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^ , Abstract. We find the smallest characteristic factor and a limit formula for the multi- 

' pie ergodic averages associated to any family of three polynomials and polynomial fam- 

1^ ■ ilies of the form {hp, hp, ■ ■ ■ , hp}- We then derive several multiple recurrence results 

, and combinatorial implications, including an answer to a question of Brown, Graham, 

and Landman, and a generalization of the Polynomial Szcmeredi Theorem of Bergelson 
and Leibman for families of three polynomials with not necessarily zero constant term. 
I We also simplify and generalize a recent result of Bergelson, Host, and Kra, showing 

that for all £ > and every subset of the integers A the set 
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{n G N: d*(A n (A n (A +p2(n)) n (A + paW)) > {d*{A))^ - s} 

has bounded gaps for "most" choices of integer polynomials pi,p2,P3. 
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c3 : 1. Introduction and main results 

1.1. Background. A far reaching generalization of the theorem of Szemeredi [32] on 
arithmetic progressions states that every subset of the integers with positive upper Ba- 
nach densit}{l| contains infinitely many configurations of the form {x, x + Pi{n), . . . , x + 
Pk{n)}, where pi, . . . ,pk 'is any collection of integer polynomials (meaning they have inte- 
ger coefficients) with zero constant term. This was proved by Bergelson and Leibman [7] 
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using a Correspondence Principle of Furstenberg and the following result in ergodic the- 
ory: 



Theorem 1.1 (Bergelson &c Leibman [7]). Let {X, X , ^,T) be an invertible measure 
preserving system and let pi, . . . ,pk be integer polynomials withpi{0) = fori = 1, . . . ,k. 
If Ae B with n{A) > 0, then 

N-l 

(1) liminf - — - V fi(A n TP^^^^A n . . . n rp^(")A) > o. 

n=M 

A key step in establishing multiple recurrence properties like the one above is to 
analyze the limiting behavior of some closely related multiple ergodic averages. For the 
previous result the relevant ones are 

1 

n=M 

Bergelson and Leibman studied these averages in [7], in a depth that was sufficient for 
proving (Q. Obtaining a better understanding of their limiting behavior (as N—M — >■ oo) 
in ^^(yu) has been a driving force of research in ergodic theory during the last two decades. 
The basic approach for studying them goes back to the original paper of Furstenberg [I5] . 
Using modern terminology, it consists of finding an appropriate factor C of a given system, 
called characteristic factor, such that the L^-limit of the averages in question remains 
unchanged when each function is replaced by its projection on this factor. Equivalently, 
this means that the averages (P) converge to in L'^{n) as — M — > cxo whenever 
E(/i|C) = for some i = 1, . . . , k, where IE(/|C) is the conditional expectation of / given 
C. The next step is to obtain a concrete description for some well chosen characteristic 
factor that is going to facilitate our study. Using methods from [19], this was done in 
[20] for weak convergence, and in [25] for strong convergence of the averages (P). 



Theorem 1.2 (Host &; Kra [20J-Leibman [25j). Let pi, . . . ,pk be a family of essen- 
tially distinct {meaning, pi and pi — pj ^ const for i ^ j) integer polynomials. Then 
there exists a d = d{pi, . . . ,pk) G N with the following property: For every invertible er- 
godic system some characteristic factor for the averages (P) is an inverse limit of d- step 
nilsystems {defined in SectionlE) ■ 

This result opens up the road for a better understanding of the limiting behavior of 
the averages (P), and in fact combined with a recent result of Leibman [22] immediately 
implies that they converge in L^(/i). But we are still left with some interesting problems 
since computing the smallest characteristic factor and the actual limit in the case of a 
nilsystem is still a difficult task. For example, it is not even clear from the results in 
[20] and [25] whether the minimal d{pi,p2) is bounded when the polynomials pi,p2 vary, 
and what the limit of the averages (P) is for k = 2. Formulas for the limit are known 



when all the polynomials are linear (see [M]) or linearly independent (see [13]). Also, 
very recently some other cases where covered in p7] . 

In this paper we are going to find the smallest characteristic factor and limit formulas 
for the averages (P) for any family of three polynomials and for polynomial families of 
the form {hp, hp, ■ ■ ■ , hp}- We will then use these results to derive several combinatorial 
implications. 

1.2. Results in ergodic theory. Given a measure preserving system and a family of 
integer polynomials P = {pi, . . . , p^} we say that a factor C is the smallest characteristic 
factor for P, if it is a characteristic factor for the averages (P) and it is a factor of 
every other such characteristic factor. We will completely determine the structure of 
the smallest characteristic factor for any family of three polynomials and the family 
{hp, hp, ■ ■ ■ ,hp}- The reader who is not familiar with the notions we use in ergodic 
theory may wish to consult Section [2?T] first. 

We first deal with the polynomial family {hp, hp, ■ ■ ■ , hp}'- 

Theorem A. Let {X, X , fi, T) be an invertible ergodic system, p be a nonconstant integer 
polynomial, and h, ■ ■ ■ ,h nonzero distinct integers. Ifk>2 then the {k — l)-step nilfactor 
Zk-i is the smallest characteristic factor for the multiple ergodic averages 



Moreover, if the system is totally ergodic then the L"^ -limit as N — M oo does not 
depend on the choice of the polynomial p and can be computed explicitly. 

We will use this result to answer a question of Brown, Graham and Landman ^ (see 
Theorem D), and to deal with characteristic factors for families of three polynomials (see 
Theorem B). The proof of Theorem A is based on Proposition 12.71 which enables us to 
compare the family {hp, hp, ■ ■ ■ , hp} with the family {hn, hn, . . . , hn}. 

Before we deal with families of three polynomials we take a moment to define three 
classes of polynomial families that will help us expedite the discussion: 

Definition 1.3. We say that the family {pi,P2,P3} of essentially distinct integer poly- 
nomials is of type (ei), (62), (63), if some permutation of the polynomials {pi,P2,P3}, 
where Pi = Pi — PiifS), i = 1,2, 3, has the form {Ip, mp, rp}, {Ip, mp, kp^ + rp}, {kp^ + 
Ip, kp^ + mp, kp^ + rp} correspondingly, for some integer polynomial p and constants 
/c, /, m, r G Z with A; 7^ 0. 



'li k — \ it is well known ([E]) that the rational Kronecker factor ICrat is a characteristic factor. 




N-l 



N-M 



n=M 
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Theorem B. Let {X,B, fi,T) be an invertible ergodic system and {^1,^25^3} be a family 
of essentially distinct integer polynomials. Consider the multiple ergodic averages 



Then the following three mutually exclusive cases describe the smallest characteristic 
factor for the averages ([2]) : 

(z) It is the rational Kronecker factor K^at if the polynomials Pi,P2iPz ^^i"^ linearly inde- 
pendent. 

[a) It is the 2-step nilf actor Z2 if the polynomials are of type (ei), and the 1-step affine 
factor A2 if the polynomials are of type (62), (63). 
(Hi) It is the Kronecker factor K. in all other cases. 

Furthermore, if the system is totally ergodic we can give explicit formulas for the L'^ -limit 
of the averages ([2]) as N — M ^ 00. 

The foUowing examples illustrate the different limiting behaviors the averages ([2]) may 
exhibit: 

(a) If P = then the rational Kronecker factor ICrat is characteristic. In the 

totally ergodic case the limit is the product of the integrals of the three functions. 

(6) If P = {n, n^, + n} then the Kronecker factor K is characteristic. In the totally 
ergodic case the limit is the same as in the case of the double averages (averaging over 
m, n) associated to the family {m, n, m + n}. 

(c) If P = {n, 2n, n^} then the Kronecker factor K, is characteristic. In the totally 
ergodic case the limit is the product of the limit of the ergodic averages corresponding 
to the family {n, 2n} and the integral of the third function. 

{d) If P = {ra, 2n, n^} then the two step affine factor A2 is characteristic. This is the 
first example that we know of a polynomial family with smallest characteristic factor 
(for totally ergodic systems) not of the form for some nonnegative integer m. In the 
totally ergodic case the limit can be computed explicitly and unlike the case {n, 2n, n^} 
it depends nontrivially on the third function. 

(e) If P = {n, 2?T,, Sra} or P = {n^, 2n^, Sra^} then the 2-step nilfactor Z2 is characteris- 
tic. In the totally ergodic case the limit is the same in both cases and can be computed 
explicitly. 

The proof of Theorem B is rather complicated so let us briefiy explain the main ideas. 
Our first step is to use Theorem 11.21 in order to show that it suffices to restrict our study 
to totally ergodic nilsystems (Proposition 14. II) . At this point we are left with establishing 
various uniform distribution properties on nilmanifolds. Our main tool is a ^^reduction 
to affine argument'^ which consists of the following two steps: (i) Reduce the uniform 
distribution problem to a simpler one that involves only nilpotent affine transformations 

"'in case (?) it is the smallest under the extra assumption that the system is totally ergodic. 




N-l 



(2) 



N -M 



n=M 
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on finite dimensional tori. This reduction is done using Theorems 12.51 and 12.61 but varies 
in difficulty depending on the problem, (ii) Verify the simplified (but often challenging) 
problem "by hand" for affine transformations. Here is a more detailed sketch of how this 
plan is executed to deal with the various parts of Theorem B: 

Part (z) deals with linearly independent polynomial families, a case that has been 
already worked out in [13| and [H] using a "reduction to affine argument". Part {ii) 
deals with families of type (ei), (62), and (63). A typical family of type (ei) is P = 
{ra^, 2n^, 3n^}. It follows from Theorem A (which is again proved using a "reduction 
to affine argument") that P ~ {?7,,2n, 3n}B This completes our task since it is known 
([To], [H]) that for this family the factor Z2 is characteristic. A typical family of type 
(62) is P = {n, 2n, n^}. To deal with this case, we first use Van der Corput Lemma and 
the fact that for families of the form {n,n'^,n'^ + kn} [k 7^ 0) the Kronecker factor /C 
is characteristic (Lemma 14.21 and 14.31) . to show that if E(/3|/C) = then the averages 
([2]) converge to zero in L^. This fact greatly simplifies the analysis, and we are led to 
consider averages corresponding to the family {n, 2n} for a transformation S = T x R 
where P is a 2-step affine transformation on T^. From this we deduce using a result from 
[T2] that the factor A2 is characteristic. To complete the study of families of type (62) 
we need also to show that {Ip, mp, kp^ + rp} ~ {/n, mn, kn^ + rn\ when the polynomial 
p is nonconstant. To do this we use Proposition 12.71 (again proved using a"reduction to 
affine argument") which roughly speaking tells us that if p{n) is a nonconstant integer 
polynomial, then the substitution n —* p{n) does not change the distribution of any 
polynomial sequence that has connected orbit closure. Families of type (es) are easily 
reduced to families of type (62), thus completing the study of part (ii). Finally, to deal 
with part (Hi) , the crucial step is Proposition 13.71 We show there that the polynomial 
families that were not covered by part (i) and (ii) have Weyl complexity 2 (defined in 
Section [3l). This fact, combined with Lemmas 14.21 and 14. 3[ allows us to conclude that in 
this case the Kronecker factor JC is characteristic. 

The following is an immediate corollary of Theorem B: 

Corollary. For any two essentially distinct polynomials and every invertible ergodic 
system, the Kronecker factor }C is characteristic for the corresponding averages (P), and 
for any three essentially distinct polynomials the 2-step nilfactor Z2 is characteristic. 

It seems plausible that for A; > 2 the {k — l)-step nilfactor Zk-^i is characteristic 
for any family of k essentially distinct polynomials. Moreover, one would expect that 
for k > 2 the smallest m for which the factor Zm~i is characteristic for a family P of 
essentially distinct integer polynomials is W{P) (defined in Section[3]). It is an immediate 
consequence of Theorem B and Proposition 13.71 that both statements hold for k = 2,3. 



We say that two polynomial families are equivalent (we write P ^ Q) if for totally ergodic systems 
the corresponding averages ([2]) have the same L^-limit. 
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Next we establish a multiple recurrence result that generalizes a result of Bergelson, 
Host, and Kra [6]: 

Theorem C. Let {X, X , fi,T) be an invertible ergodic system, X with fi{A) > 0, 
and {pi,P2,P3} be integer polynomials with Pi{0) = 0, for i = 1,2,3. Then for every 
e > the set 

{neN: /i (A n TP^(")A n T^'^Wa) > fx{Af - e} 
has bounded gaps. Moreover, the set 

{neN: fi{An Tpi(")A n TP^^A n TP^(")A) > fi{Ay - e} 

has bounded gaps, unless the polynomials are essentially distinct and of type (ei) with 
I < m < r and r ^ I + m, or of type (62), (63). 

This result was established in |6] for the polynomial families {n, 2n} and {n, 2n, 3n}. 
Moreover, it was shown that an analogous result fails for the family {n, 2n, 3n, 4n}, in fact 
no fixed power of ii{A) works as a lower bound. To prove Theorem C we use Theorem A 
and parts (i), {Hi) of Theorem B. We remark that even for the two cases covered in 
our argument is different and much simpler (1 and 2 pages long correspondingly). The 
crucial observation is that although we cannot get good lower bounds for the averages 
corresponding to the families {n, 2n} and {ra, 2r;,, 3n} if we average over the full set of 
positive integers, we can get optimal lower bounds as long as the average is taken over 
an appropriately chosen subset of the integers (that depends on the system given) j§ This 
observation greatly simplifies the whole analysis, as we do not have to rely on the rather 
complicated nilsequence decompositions used in |6]. 

For the exceptional polynomial families of Theorem C we believe that the analogous 
result fails and we provide conditional counterexamples in Section [531 

1.3. Results in combinatorics. We are going to utilize the previous results in ergodic 
theory to derive several implications in combinatorics. We mention them in increasing 
degree of difficulty. 

We start with an answer to a question of Brown, Graham, and Landman. In [9] the 
authors define a set S" C Z to be large if every finite coloring of the positive integers 
contains arbitrarily long monochromatic arithmetic progressions with common difference 
a nonzero integer in S. It follows from Theorem 11.11 that if p is an integer polynomial with 
pifS) = then the set Sp = {p{n) : n G N} is large. If we do not assume that p(0) = an 
obvious necessary condition for the set Sp to be large is that it contains multiples of every 
positive integer. The authors of [Hj asked whether this condition is also sufficient and in 
particular whether the range of the polynomial p{n) = {n^ — 13) (n^ — 17) (n^ — 221) is 

^This is best exemplified by considering an irrational rotation a € T. Although it is not possible to 
get good lower bounds for the average of the sequence ijl{A n (A + na) n (A + 2na)) when n ranges over 
N, if we restrict the range of n to the set Ss = {n: {na} < 5}, it is easy to show that for every e > if 
5 is small enough then the average of the corresponding subsequence is greater than h^AY — e. 
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large; this is an example of a polynomial with no linear integer factors whose range does 
contain multiples of every positive integei@ (this can be easily verified using properties of 
the Legendre symbol). We will give a positive answer to these questions, in fact we will 
verify a stronger "density" statement. We say that C Z is a set of multiple recurrence 
if every subset of the integers with positive density contains arbitrarily long arithmetic 
progressions with common difference a nonzero integer in 5*. We show: 

Theorem D. Let p be an integer polynomial. Then Sp = {p{n) : n & N} is a set of 

multiple recurrence if and only if it contains multiples of every positive integer. 

To prove this result we use Theorem A and Furstenberg's Multiple Recurrence The- 
orem [15]. Polynomials that satisfy the conditions of Theorem D have been studied in 
jl]. It is shown there that p{n) = (mod m) is solvable for every m G N if and only if 
it is solvable for a finite set of m G N explicitly depending on p. 

Our next application is to construct a set 5* that has bad recurrence properties but its 
set of squares S"^ is a set of multiple recurrence. Note that if S" is a set multiple recurrence 
it is not known whether its set of squares S"^ is always a set of multiple recurrence (the 
chromatic version of this question was conjectured to be true in [9]). 

Theorem E. There exists a set S G N that is not a set of multiple (in fact not even 
single) recurrence but p{S) = {p{s),s G S} is a set of multiple recurrence for every 
integer polynomial p with degree greater than 1. 

Our example is exphcit, in fact we show that the set S* = {n G N: {n\/2} G [1/4, 3/4]} 
works. To prove this we rely on Lemma 12.81 

Our next application deals with an extension of Theorem ll.ll to families of polynomials 
with not necessarily zero constant term. We say that the family of integer polynomials 
{pi, . . . ,Pk} is universal if every subset of the integers with positive density contains 
infinitely many configurations of the form {x, x + pi{n), . . . ,x + Pk{n)}, where x,n gN. 
From Theorem 11.11 we know that every family of integer polynomials with zero constant 
term is universal. We show: 

Theorem F. The family of integer polynomials {pi,P2,P3} is universal if and only if 
the congruence pi{n) = P2{n) = pz{n) = (mod m) has a solution for every m eN. 

To prove this result we make essential use of Theorem B, so we are currently unable 
to extend it to deal with families of k polynomials for k > A. 

Finally, using a modification of the Correspondence Principle of Furstenberg, due to 
Lesigne (see Section [5]), we give the following combinatorial implication of Theorem C: 



As shown in the smaUest possible degree of a polynomial having this property is 5, an example 
is p{n) = (n^ - 19)(n^ + n + 1). 
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Theorem C. Let A C N and Pi,P2,P3 be integer polynomials with Pi{0) = for i = 
1, 2, 3. Then for every e > the set 

{neN: d*(An (A + piH) n (A + P2H)) > K(A))3-e} 

has bounded gaps, and the set 

{neN: rf*(An(A + pi(n))n(A + p2H)n(A + p3H)) > {d*{A)Y-e} 

has bounded gaps, unless the polynomials are essentially distinct and of type (ei) with 
I < m < r and r ^ I + m, or of type (62), (63). 

Examples of random sets show that the lower bounds given are tight. The same 
result was established in [6j in the special case of the polynomial families {n, 2n} and 
{n,2n,3n}. In the case of the family {n, 2n} a related finite version of this result was 
established by Green [18]. Some other examples of eligible 3-term polynomial families 
are the following: {n, 3n, 4n}, {n'^, 2n^^ Sn'^} for all G N, {n, n^, an^ + hn} with a 7^ 0, 
and {n,2n, n'^} for all A; > 3. It was shown in [6] that similar lower bounds fail for the 
polynomial family {n, 2n, 3n, 4n}. In contrast to this, similar lower bounds hold for any 
family of k linearly independent polynomials with zero constant term (see |14]). 

As was the case with the corresponding result in ergodic theory, for the exceptional 
polynomial families of Theorem C we believe that the analogous result fails and we 
provide conditional counterexamples in Section [5751 

Notation: The following notation will be used throughout the article: Tf = f oT, 
e{x) = e^™, {x} = X — [x], UD-lim(a„) = if for every e > we have d*{{n: |a„| > 
e})=0. 

Acknowledgements. The author would like to thank B. Kra for helpful discussions 
during the preparation of this article, M. Johnson for helpful remarks, and S. Leibman 
for providing the simple proof of Proposition 12.71 

2. Background in ergodic theory and nilsystems 

2.1. Ergodic theory background and notation. Background information we assume 

in this article can be found in the books [16], [30], [33]. By a measure preserving system 

(or just system) we mean a quadruple (X, X, fi,T), where (X, X, fi) is a probability space 

and T : X — > X is a measurable map such that Ai(T-M) = /i(v4) for all Ae X. Without 

loss of generality we can assume that the probability space is Lebesgue. A factor of a 

system can be defined in any of the following three ways: it is a T-invariant sub-a-algebra 

V of X, it is a T-invariant sub-algebra JF of L°°{X), or it is a system (F, 3^, z/, 5) and 

a measurable map it : X' Y' , where X' is a T-invariant set and Y' is an S*- invariant 

set of full measure, such that fi o tt"^ = u and S o tt{x) = tt o T{x) for x G X'. . In a 

slight abuse of terminology, when any of these conditions holds, we say that Y (or the 

appropriate cr-algebra of ^Y) is a factor of X and call tt: X' — > the factor map. If 

the factor map tt : X' — > F' can be chosen to be injective, then we say that the systems 
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{X, X , fi,T) and {Y,y,h',S) are isomorphic (bijective maps on Lebesgue spaces have 
measurable inverses). 

If 3^ is a T-invariant sub-a-algebra of X and / G L'^ifi), we define the conditional 
expectation E(/|3^) of f with respect to y to be the orthogonal projection of / onto 
L'^{y)- We frequently use the identities 

j Eif\y) dfx = j f dfx, TEif\y) = EiTf\y). 

For each r G N, we define JCr to be the factor induced by the algebra 

{/GL-(/i):T7 = /}. 

We define JCrat to be the factor induced by the algebra generated by the functions 

{/ G L°°(/i) -.rf = f for some r G N} . 

The Kronecker factor /C is induced by the algebra spanned by the bounded eigenf unctions 
of T, i.e. functions that satisfy Tf = e{a) ■ f for some a G M. We also define higher order 
eigenfunctions and their corresponding factors. Let Sq denote the set of eigenvalues of T 
and for A; G N we define inductively 

£, = {feL^ifi):\f\ = l and T/ ■ / G £k-iiT)}. 

We call the factor spanned by Sk the k-step affine factor of the system, and denote 
it by Ak- The reason for this notation is that for totally ergodic systems the factor 
system induced by Ak is isomorphic to a nilpotent A;-step affine transformation on some 
connected compact abelian group (this is a result of Abramov [1]), and Ak is the largest 
factor with this property. 

The transformation T is ergodic if /Ci consists only of constant functions, and T is 
totally ergodic if ICrat consists only of constant functions. Every system {X, X , fi,T) 
has an ergodic decomposition, meaning that we can write fi = j fit dX{t), where A is 
a probability measure on [0, 1] and fit are T-invariant probability measures on {X, X) 
such that the systems (X, X,fit,T) are ergodic for t G [0, 1]. We sometimes denote the 
ergodic components by {Tt)t£iQ,i]- 

We say that the system {X, X , fi,T) is an inverse limit of a sequence of factors 
(X, Xj, /i, T) if {A'jjieN is an increasing sequence of T-invariant sub-cr-algebras such that 
VjeN = up to sets of measure zero. 

Following [T9j, for every system {X, X , fi,T) and function / G L°°(fj,), we define in- 
ductively the seminorms |||/|||a; as follows: For A; = 1 we set |||/|||i = |E(/|X)|0 where X is 



'''in tlSj the authors work with ergodic systems, in which case |||/|||i = / / d/i, but the whole discussion 
can be carried out for nonergodic systems as well without extra difficulties. 
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the (T-algebra of T-invariant sets. For k > 2 we set 

1 _ 

(3) lll/llir/ = lim -5^|||/-TV||ir. 

n=0 

It was shown in [12] that for every integer > 1, ||| ■ 1^ is a seminorm on L°°{fi) and 
it defines factors Zk-i in the following manner: the T-invariant sub-cr-algebra Zk-i is 
characterized by 

for / G E(/|Zfe_i) = if and only if |||/|||fc = 0. 

We remark that if (Tt)teio,i] are the ergodic components of the system then K{f\Zk(T)) = 
if and only if K{f\Zk(Tt)) = for a.e. t G [0, 1]. For ergodic systems the factor Zq 
is trivial, Zi = Ai = /C, and Ak C Z^ (the inclusion is in general proper for k > 2). 
The factors Zk are of particular interest since they are characteristic for L^-convergence 
of ergodic averages (P). Moreover, in [19] it was shown that the factor Zk is an inverse 
limit of /c-step nilsystems which brings us to our next topic of discussion. 

2.2. Nilsystems, definition and examples. Fundamental properties of nilsystems, 
related to our discussion, were studied in [2J, [29], [28], [23], and [Slj. Below we summarize 
some facts that we shall use, all the proofs can be found in [22] . 

Given a topological group G, we denote the identity element by e and we let Gq denote 
the connected component of e. li A,B C G, then [A, B] is defined to be the subgroup 
{[a,b] : a E A,b E B} where [a,b] = aba^^b^^. We define the commutator subgroups 
recursively by Gi = G and Gk+i = [G,Gk]- A group G is said to be k-step nilpotent 
if its {k + 1) commutator Gk+i is trivial. If G is a k-step nilpotent Lie group and F 
is a discrete cocompact subgroup, then the compact space X = G/T is said to be a 
k-step nilmanifold. The group G acts on G/T by left translation and the translation by 
a fixed element a G G is given by Ta{gT) = {ag)T. Let m denote the unique probability 
measure on X that is invariant under the action of G by left translations (called the Haar 
measure) and let Q /T denote the Borel a-algebra of G/T. Fixing an element a G G, we 
call the system {G/T,Q/T,m,Ta) a k-step nilsystem and call the map a nilrotation. 

If if is a closed subgroup of G then Y = (HT) /F ~ H/ {H fl F) may not be compact 
in general (take X = M/Z and H = {t\^: t G K}), but if Hx is closed in X for some 
X E X, then it can be shown that Y is compact and the set Hx can be given the structure 
of a nilmanifold. In particular if x = gT for some g E G we have Hx ~ H/A where 
A = H n gVg^^. We call any such set a sub-nilmanifold of X. 

Examples of nilsystems are rotations on compact abelian Lie groups, and more gener- 
ally, every nilpotent afiine transformation on a compact abelian Lie group is isomorphic 
to a nilsystem (see Example 1). But these examples do not cover all the possible nilsys- 
tems (see Example 2). 
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Example 1. On the space G = Z x M?, define multiplication as follows: 

if gi = {nil, xi,X2) and g2 = {ni,yi, 1/2), let 

91-92 = {mi + ni,xi + yi,X2 + 2/2 + rriiyi). 

Then G is a 2-step nilpotent group and the discrete subgroup F = is cocompact. If 

a = (mi, Oi, 02), it turns out that is isomorphic to the a nilpotent affine transformation 
5; T2 _^ T2 giygj^ 

S{xi, X2) = {xi + ai, X2 + mixi + 02). 

Example 2. On the space G = R'^, define multiplication as follows: 

if 9i = {xi,X2,X3) and 92 = ?/2, 1/3), let 

91-92 = {xi + yi, X2 + y2, X3 + y3 + xiy2). 

Then G is a 2-step nilpotent group and the discrete subgroup F = Z'^ is cocompact. Let 
a = (ai,a2,0), where ai,a2 G [0,1) are linearly independent. It turns out that is 
isomorphic to a skew product transformation 5 : T'^ ^ that has the form 

S{Xi,X2) = {Xi + ai,X2 + a2,X3 + f{Xi,X2)), 

where / : ^ T is defined by 

f{xi,X2) = {xi + ai)[x2 + 02] - Xi[x2] - aiX2. 

It can be shown that S (or Tq) is not isomorphic to a nilpotent affine transformation on 
some finite dimensional torus. 

Let {X = G/T, Q/T, m, To) be an ergodic nilsystem. The subgroup < Go, a > projects 
to an open subgroup of X that is invariant under a. By ergodicity this projection equals 
X. Hence, X =< Go, a > /V where F' = Ffl < Go, a >■ Using this representation of X 
for ergodic nilsystems we have that 

(4) G is generated by the connected component of the identity element and a. 

/^From now on when we work with an ergodic nilsystem we will freely assume that 
hypothesis (jlj) is satisfied. We remark that under this hypothesis it was shown in p3] 
that for every integer k > 2 the group Gk is connected. 
We will make frequent use of the following simple facts: 

Proposition 2.1. Let {X = G/F, ^/F, m, T^) be an ergodic nilsystem. Then 
(i) The system is totally ergodic if and only if X is connected. 

(a) There exists an r eN such that the {finitely many) ergodic components of T^ are 
totally ergodic. 

Proof. We first prove statement (z). Suppose that the system is totally ergodic. Let Xq 

be the identity component of X. Since X is compact, it is a disjoint union of a finite 

number of translations of Xq. Since a permutes these copies, there exists an r G N such 
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that preserves Xq. By assumption the translation by T^r = is ergodic and so 
Xo = X. 

Conversely, suppose that X is connected and let r G N. Because Ta is ergodic, there 
exists Xq & X such that the sequence {a"vr(a;o)}nGN is uniformly distributed in Z = 
G/{[G, G]T), where n: X Z is the natural projection. Since Z is a connected compact 
abelian group, it is well known that {a'""'vr(xo)}neN is also uniformly distributed in Z. 
By Theorem 12.51 below we have that = T^r is ergodic. Since r G N was arbitrary, 
is totally ergodic. 

We now prove statement (n). The Kronecker factor of an ergodic nilsystem is isomor- 
phic to a rotation on a monothetic compact abelian Lie group G. Every such group has 
the form x T'^^ for some positive integer di and nonnegative integer d2, where 
denotes the cyclic group with d elements. It follows that JCrat = ^d^, and T'^^ has finitely 
many ergodic components and they are all totally ergodic. □ 

2.3. Factors of nilsystems. Given an ergodic nilsystem the following result allows us 
to identify its factors Zk{Ta): 

Theorem 2.2 (Ziegler ^SSj). Let {X = G/r,Q/r,m,Ta) be an ergodic nilsystem. Then 
f G Zk{Ta) if and only if f E L°°{rn) and f factors through G/ (Gk+iT). 

It will also be convenient for us to identify the 2-step affine factor A2 of an ergodic 
nilsystem. We adapt a technique from |23] to do this. We first need a lemma: 

Lemma 2.3. Let {X = G/r,Q/r,m,Ta) be an ergodic nilsystem. If f & ^2{Ta) then for 
every b E G we have fb G £^2(^0); where fb{x) = f{bx). 

Proof. We know that A2{Ta) is a factor of Z2{Ta), so by Theorem 12.21 the function / 
factors through G/{G^V). Hence, after replacing G by G/G3 we can assume that G is 
2-step nilpotent. We know from [1] that |/| = const, so we can assume that |/| = 1 in 
which case we have that / = f~^. Since / G £2{Ta) there exists h G £i{Ta) such that 

f{ax) = h{x) ■ f{x). 

By Theorem 12.21 the function h factors through the compact abelian group G/{[G, G]r). 
Moreover, since h is an eigenfunction of Ta it is a character of G. 
We first claim that 

(5) if cG [G,^] then/, = A,-/ 

for some constant Ac G C. Since h{cx) = h{x) and c belongs to the center of G we find 
that 

Tafc = f{cax) = f{acx) = h{cx) ■ f{cx) = h{x) ■ f{cx) = h ■ f^. 

Hence, fc-fE Si{Ta). We define a map (p: [G, G] —>■ Si{Ta) by 0(c) = fc ■ f- It suffices 

to show that (j){[G,G]) C C, where C is the set of constant functions. We will use a 

connectedness argument to do this. If we equip Si{Ta) with the L'^{m) topology then 
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the map (j) is continuous. Since is ergodic the connected component of the function 1 
in £i{Ta) is the set C . Since is continuous, 0(e) = 1, and [G, G] is connected, we have 
that 0([G, G]) C C. This proves the claim. 

We now show that for every b E G we have fb G £^2(^0)- We compute 

(6) fb{o,x) = fipax) = f{ab[a,b]x) = h{b[a,b]x) ■ f{b[a,b]x). 
Since h is a character of G we have 

(7) h{b[a,b]x) = Xih{x). 
Using that [a, b] belongs to the center of G and ([5]) we find 

(8) f{b[a,b]x) = f{[b,a]bx) = f^^MM = X2f{bx). 
Putting together equations ([7j), ([H]), we find 

Tafb = hi - fb 

where hi = XiX2h E Si{Ta). This completes the proof. □ 

Proposition 2.4. Let {X = G/F, ^/F, m, T^) be an ergodic nilsystem and suppose that 
X is connected. Then f G A2{Ta) if and only if f E L°°{m) and f factors through 
G/{G3[Go,Go]r). 

Proof. Suppose first that / E L°°{m) factors through G/{Gs[Go,Go]T). Replacing G 
by the group G/{G3[Go,Go]) we can assume that G is 2-step nilpotent and that Go is 
abelian. In this case, by Theorem 12.61 below the system is isomorphic to a 2-step nilpotent 
affine transformation on some finite dimensional torus. For such systems it is easy to 
verify that A2{Ta) = L°°(m), and so / G 

We move now to the converse. It suffices to show that if / G £2{Ta) then / factors 
through G/(G3[Go, Go]F). We know from [Ij that / = const, so we can assume that 
I/I = 1 in which case we have that / = f~^. Since ^2(^0) is a factor of Z2{Ta), by 
Theorem 12.21 the function / factors through G/(G3F). So it remains to show that / 
is invariant under elements in [Go,Go]. We define the map 0: G ^ L°°(m) by 0(6) = 
f{bx) ■ f{x). We need to show that 0([Go,Go]) = 1. First notice that by Lemma [2731 
the map takes values in S2{Ta). Next we claim that 0(Go) C G where G is the set 
of constant functions of modulus 1. We will use a connectedness argument to show this 
(similar to the one used in Lemma [231) . If we equip Si{Ta) with the L'^{m) topology then 
the map is continuous. The connected component of the function 1 in £^2(^0) is the set 
G. One can see this by using the fact that if / G £^2(^0) is nonconstant then J f dm = 
(see m), which implies that ||/ — c||^2(^) = for c G G. Since is continuous and 
0(e) = 1 we have that 0(Go) C G. Now it is easy to check that 0: Gq — > G satisfies 

0(a6) = 0(a) ■ 0(6) = 0(6a), 0(a6) = 0(6a) ■ 0([a, 6]), a,b E Go, 
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which imphes (obviously </)(x) 7^ for x G G) that 

0([a, b]) = 1, a,b E Gq. 
This completes the proof. □ 

2.4. Polynomial sequences on nilmanifolds. If G is a nilpotent Lie group, ai, . . . , G 
G, and pi, . . . ,pk are integer polynomials N'^ ^ Z, a sequence of the form 

is called a polynomial sequence in G. If the polynomials pi{n) , . . . , pk{n) are linear 
then g{n) is called a linear sequence. The following result of Leibman ([23], [21]) gives 
information about the orbit closure of polynomial sequences on nilmanifolds and helps us 
handle their uniform distribution propertied by reducing them to uniform distribution 
properties on a certain factor: 

Theorem 2.5 (Leibman [23], [23]). Let X = G/T be a nilmanifold and g{n) be a 
polynomial sequence in G. Define Z = G/{[Go,Go]T) and Z' = G/{[G,G]T) and let 
n: X —>■ Z and n' : X ^ Z' be the corresponding natural projections. Then for every 
X G X: 

(i) There exist sub-nilmanifolds Yi = Hxi of X, where H is a closed subgroup of 
G {depending on x) and xi,...,Xk G X, such that {g{n)x}^^^d = Uj=i and for 
i = 1, . . . , k the sequence {g{kn + i)}nm is uniformly distributed on Yi. If g{n) = a" for 
some a E G, then k = 1. 

(a) If X is connected then the sequence {g{'n)x}^^^d is uniformly distributed in X if 
and only if it is dense in X if and only if {g{n)n{x)}nem is dense in Z . 

(Hi) If X is connected and Oi, . . . , G G are commuting elements that together with 
Gq generate G, and g{n) = a^^*-"^ ■ ■ ■ a^*"*-"^ is a linear sequence, then {gitijxj^^^d is 
dense in X if and only z/ {(7(n)7r'(s)}ngj^d is dense in Z' . 

We remark that the groups Go, [Go, Go], and [G.G] are normal subgroups of G. Also 
note that the connected component the identity element of the group G/[Go,Go] is 
abelian. The next result shows how we can use this property to our advantage. In order 
to state it we need some notation. If G is a group then a map T: G ^ G is said to be 
affine if T{g) = bA{g) for a homomorphism A of G and some b E G. The homomorphism 
A is said to be k-step nilpotent if there exists A; G N so that [A — \dY = 0. 

Theorem 2.6 (Prantzikinakis &: Kra [13]). Let X = G/T be a connected k-step 
nilmanifold such that Go is abelian. Then every nilrotation Ta{x) = ax defined on X with 
the Haar measure m is isomorphic to a k-step nilpotent affine transformation on some 
finite dimensional torus. Furthermore, the conjugation can be chosen to be continuous. 



^For the purpose of this article, we will say that a sequence {x(n)}„gpjd is uniformly distributed on 
the nilmanifold X with the Haar measure m, if for every F0lner sequence <I>Ar in N"^ and continuous 
/ : X ^ C, we have limAr^oo ]^ EnG*« /(^^(n)) = J f dm. 
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Next we give two applications of Theorems 12.51 and 12.61 that will be needed in the 
sequel. We will use the first one frequently, for example in the proofs of Theorems A, B 
and F. The simple argument given below was communicated to us by S. Leibman: 

Proposition 2.7. Suppose that X = G/T is a nilmanifold, g: N —>■ G is a polyno- 
mial sequence, and x & X is such that Y = {g{n)x}^^^ is connected. Then for every 
nonconstant integer polynomial p we have Y = {g{p{n))x}^^^. 

Proof. Since Y is connected we have by part (i) of Theorem 12.51 that Y is isomorphic to 
a subnilmanifold H/A of X. Hence, we can assume that Y = H/A. By Theorem 12.51 
it suffices to show that the sequence {g{p{n))7i{x)}n£n is uniformly distributed on Z = 
H/{[Hq, Hq]A) where vr: F — > Z is the natural projection. Substituting H/[Hq, Hq] for 
H we can assume that Hq is abelian. Suppose that g{n) = a^^''"''a2^*-"^ ■ ■ ■ a^''^"'^ where 
ai, . . . , ttfe G G. Since Y is connected and Hq is abelian, by Theorem 12.61 we can assume 
that Y = and the nilrotations Ta^, i = 1, . . . , k, are nilpotent affine transformations 
on T"*. Then the coordinates of the sequence {g{n)x}n£n are polynomials in n with real 
coefficients, and our problem reduces to the following one: If u : N ^ T"* is a sequence 
with polynomial coordinates such that {u{n)}^^^ = T™, then {u{p{n))}^^^ = T"^ for 
every nonconstant polynomial p. To see this, first notice that u has the form 

u{n) = UQ{n)q + Ui{n)ai + . . . + ui{n)ai 

where Ui are integer- vector- valued polynomials, g G Q, and ai, . . . , are linearly inde- 
pendent irrational numbers. Then using Corollary 2.4 in [8] we have that u{n) is dense 
in T"^ if and only if 

Span('Ui(n)) + . . . + Span('Ufc(n)) (mod 1) = T™, 

where for u{n) = {qi{n) , . . . qr{n)) we define Span('u(?2)) = Spa.n{{qi{x) , . . . qr{x)) , x G 
R}. But clearly the last identity remains valid if we replace n with any nonconstant 
polynomial p{n). This completes the proof. □ 

The next lemma will be used in the proof of Theorem E. 

Lemma 2.8. Suppose that X = G/T is a nilmanifold, g: N ^ G is a polynomial 
sequence, and p is an integer polynomial with degp > 1. Then for every x G X and 
/3 G T irrational we have 

(9) {{np, c/(p(n))x)}„gj^ = T x {g{pin)x)}^^^. 

Proof. Suppose first that the set Y = {g{p{'n)x)}^^^ is connected. Working with the 
sequence {/i(n)}„gN = {{nP, g{n)x)}neN on T x F and repeating the argument used in 
the previous lemma we can reduce our problem to the following one: If m : N — is a 
sequence with polynomial coordinates such that {u{p{n))}^^^ = T*" and degp > 1, then 
{(n/?, M(p(n))}„gp5j = 1''"+-'^. To see this, first notice that u has the form 

u{n) = uo{n)q + ui{n)ai + . . . + ui{n)ai 
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where Ui are integer- vector- valued polynomials, g G Q, and ai,...,afc are rationally 
independent numbers. Since {u{p{n))}^^^ = T™, by Corollary 2.4 in [8j we have 

(10) Span(Mi(p(n))) + . . . + Span{uk{p{n))) (mod 1) = T™. 

We also have 

(n/?, g{p{n))x) = {n, 0)P + (0, Uoip{n)))q + (0, Mi(p(n)))ai + . . . + (0, Ui{p{n)))ai. 

Since the polynomials Ui{p{n)), i = 1, . . . ,1, have degree greater than 1 it is easy to check 
that the set {{nP,u{p{n)))}^^^ is equal to 

Span((r;,, 0)) + Span((0, Ui{p{n)))) + . . . + Span((0, Uk{p{n))) (mod 1), 

and by ffTOj) this is equal to Y"^^^ . This completes the proof. 

In the general case we argue as follows: By [23j there exists an r G N such that 
{g{p{rn + 0)}neN connected for i = 0, . . . , r — 1. Repeating the previous argument for 
the sequence {h{rn + z)}„gN "we find that 

{h{rn + = T x {g{j){rn + i))x)}^^^ 

for i = 0,...,r — 1. This implies ^ and completes the proof. □ 

2.5. Limit formula for linear sequences. In the case where all the polynomials are 
linear the limit of the corresponding multiple ergodic averages (P) was computed in |t34j 
(for a simpler proof see [6j). To state the result we need some notation. Let G/F be a 
nilmanifold. Given /i, . . . , G N define the set 

( h ) (Ik) (Ik) ( h ) 

^it) = if a < 6) and let A = T'^ H H . It was shown in [22] that i7 is a closed subgroup 
of G'^. The discrete subgroup A is cocompact so the nilmanifold H/A carries a Haar 
measure, call it mn- The next result is a straightforward generalization of a formula 
given by Ziegler in [34j that can be obtained using some computations of Leibman in 

m- 

Theorem 2.9 (Ziegler [34]). Let {X = G/r,Q/r,Ta,m) be an ergodic nilsystem and 
li, . . . Jk & If fi, ■ ■ ■ , fk ^ L°°{X) then for a.e. x = gV & X we have 

N-l 

hm ^^^/,(Ti-x)-..../fc(Ti^"x)= / f^{gy^Ty....fkigykT)dmj,iyA), 

n=M JHjlX 

where y = {yi, . . . , y^), and H , A are as before. 

Combining this with Theorem 12.51 we easily deduce the following: 

Corollary 2.10. Let {X = G/F, ^/F, T^, m) be an ergodic nilsystem with X connected 
{or equivalently Ta is totally ergodic) and /i, . . . ,4 G Z. Then for a.e. x E X the set 
Hx = {{a''^^x, a'2"a;, . . . , a'*"'a;)}„gN ^■^ connected. 
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Proof. By Theorems 12 . 5 1 and 12 . 9 1 we have that for a.e. x & X the set is homeomorphic 
to the nilmanifold H/A where the subgroup H and A are as before. Since X = G/T is 
connected and Gi is connected for i > 2 it follows that H/ A is connected. Hence, is 
connected for a.e. x E X. □ 



3. Weyl complexity for families of three polynomials 

Following ^ , we will define the Weyl complexity of a family P = {pi , . . . , } of essen- 
tially distinct integer polynomials. Roughly speaking, this notion is designed to capture 
the minimum m G N for which the factor Zm-i is characteristic for the corresponding 
ergodic averages (P). In Proposition 13 . 71 we will give an effective way of determining the 
Weyl complexity of any family of three polynomials. 

3.1. Definition of Weyl complexity and basic properties. A connected Weyl sys- 
tem is a system induced by an ergodic nilpotent affine transformation acting on some 
finite dimensional torus with the Haar measure. A standard Weyl system of level d is a 
system induced by a transformation T: T'^ — ^ T*^ given by 

(11) T{xi, . . . , Xd) = {xi + a, X2 + xi, . . . , Xd + Xd-i) 

for some irrational number a G T. A quasi- standard Weyl system of level ci is a system 
induced by a transformation T : T'^ — > T'^ given by 

d-l 

(12) T(xi,X2, . . . , Xd) = (xi + ai,X2 + m2,iXi + a2, ■ ■ ■ ,Xd + '^'mdjXj + ad) 

where G T, ai is irrational, and nii^i-i ^ for all i = 2, . . . ,d. Note that every 
quasi-standard Weyl system is ergodic ([T6j page 67). 

Given a system (X, T) we denote the diagonal in X^~^^ by A^fc+i , and we define the 
orbit of a polynomial family P = {pi, . . . ,pk} with respect to the system {X, T) to be 

0(P,Axfc+i,T) = {(s,T^'^(")a;,...,TP'=(")x): x G X,n G N}. 

Definition 3.1. Let P = {pi, . . . ,pk} be a family of distinct integer polynomials with 
Pi{0) = for i = 1, . . . , A;. The Weyl complexity W{P) is the minimal r G N with the 
following property: For every d E N with d > r, for some/ever}l§ quasi- standard Weyl 
system {X,T) of level d we have 

(13) {(0, . . . , 0, x,, . . . , Xd)}"^' C 0(P, A^.+i, T). 

For a general family P = {pi, . . . ,pk} of essentially distinct polynomials we define 
W{P) = W{p, - pi(0), ...,Pk- PkiO))- 

^It was shown in f8] that if (fT3|) holds for some quasi-standard Weyl system of level d then it holds 
for all. 
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The next two results give equivalent characterizations of the Weyl complexity that are 
better suited for our purposes. The first follows easily from the definition. 

Proposition 3.2. The Weyl complexity W{P) of a family P = {pi, . . . ,pk} of essentially 
distinct integer polynomials is the maximal s E N {or 1 if there is no such s) with the 
following property: For some/every quasi- standard Weyl system {X,T) of level s — 1 of 
the form ( IT2l) . there exist characters Xi of X , i = 0, . . . , k, at least one of which depends 
nontrivially on the variable Xs~i, such that 

Xo(x)-Xi(T^^^"^a;)-...-Xfc(T^'^(")x) = l 

for every x G T^~^. 

For a proof of the next result see the remarks after Proposition 5.1 in [8]. 

Proposition 3.3. The Weyl complexity W{P) of a family of essentially distinct integer 
polynomials P = {pi, . . . ,pk} is the minimal m G N with the following property: For 
every connected Weyl system {X,T) the factor Z^-i is characteristic for L'^ -convergence 
or weak convergence of the averages (P) . 

We remark that for a quasi-standard Weyl system of the form (fT2l) the factor Zm 
coincides with the sub-cr-algebra of sets that depend only on the first m coordinates. 
We will make frequent use of the following simple identity: 

Proposition 3.4. // {pi, . . . is a family of essentially distinct polynomials then 

W{pi,p2, ...,pk) = W{pi -pk,P2 -Pk,-- ■,Pk-i -Pk, -Pk)- 

Proof. This follows immediately from Proposition 13.31 and the identity 

□ 

3.2. Different scenarios for the Weyl complexity of three polynomials. We will 

give an explicit criterion for determining W{pi,p2,pz)- We first show: 

Proposition 3.5. // {piiP2iP'i} is a family of essentially distinct polynomials then 

W{pi,P2,P3) < 3. 

Proof. We argue by contradiction. Suppose that W{pi,p2,P3) > 4. We can assume that 
Pi{0) = for i = 1,2,3. Consider the quasi-standard Weyl system (T^,T) where 

T{xi,X2, xs) = (xi + a, X2 + 2xi + a,Xs + Sxi + 3x2 + a). 

By Proposition l3.2l there exist characters xo, Xi; X2, Xs of T^, at least one of which depends 
nontrivially on the variable 0:3, such that 

(14) xo{x) ■ XiiT^'^^'^x) ■ X2(TP^(")x) ■ X3(T^^^"^a;) = 1 
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for all X eT^. We use that 



T"'(xi, X2, X3) = (xi + na, X2 + 2nxi + ri^a, X3 + 3nx2 + Sn^xi + n^a) 
and substitute in f|T4l) . Suppose that 

Xi{xi,X2, X3) = e{kiXi + liX2 + rriiX^) 
for some integers /cj, /j, rrii for i = 0, 1, 2, 3. Plugging in f[T^ we get that the system 

(15) fcipi + I1P2 + "^iPs = 

(16) 'ikipl + + 3mip2 + 2A;2Pi + 2/2^2 + 2m2P3 = 

(17) kip\ + hpl + mip^ + fcaPi + /2P2 + ^2pI + ^sPi + ^3^2 + "^sPs = 

has a solution on the integers ki,li,mi, i = 1,2,3, with at least one of the /ci,/i,mi 
nonzero. Let di = degpi, i = 1,2,3, and ai,bi,Ci be the leading coefficients of the 
polynomials Pi,P2,P3- After rearranging the polynomials we can assume that di > d2 > 
d^. We consider three cases: 

Case 1. If di > d2 > d^ then ( |T5i) gives ki = 0, so d2 = ds = d > 0. If = then 
looking at the leading coefficients of the polynomials in ( |T6i) we get that h = —mi which 
implies (using ( |T5l) ) that p2 = ps, a contradiction. Hence, k2 7^ and since h 7^ —mi we 
get from ( fT6l) that di = 2(i. But then the polynomial on the left hand side of ( |T7I) has 
degree 4(i, a contradiction. 

Case 2. If di > d2 > ds then using equations ( |T5l) . ( |T6|l (which are the same as 
( fT9l) . ( l20|) ). and arguing as in Case 2 of Proposition 13.71 below, we derive that pi = P2, a 
contradiction. 

Case 3. U di = d2 = d^ = d then looking at the leading coefficients of the polynomials 
in ([ISD, ([16]), (dZD, we get that the system 

fcifli + /161 + miCi = 
kiol + lih\ + miCi = 
kia\ + lih\ + mic"^ = 

has a nontrivial integer solution on A;i,Zi,mi. The determinant of the corresponding 
matrix is ai6iCi(ai — 6i)(6i — ci)(ci — ai). Since ai, 61, ci are nonzero, two of them must 
be equal. Without loss of generality we can assume that ai = hi. Then after replacing 
Pi with qi = —pi, P2 with q2 = P2 — Pi, Pz with qz = Pz — Pi, and using Proposition 13.41 
reduces our problem to either Case 1 or Case 2. So we get again a contradiction showing 
that Vr(pi,P2,P3) <3. □ 

We will also need the following simple lemma: 
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Lemma 3.6. Suppose that ai,6i,ci G Z are nonzero and distinct, a2,b2,C2 G Z, and 
kiai + libi + miCi = 0, kia2 + /1&2 + ^iC2 = 

fcia^ + libl + micl = 0, kiaia2 + /1&1&2 + miCiC2 = 0, 

/or some integers fci, /i, mi, not a// of them zero. Then there exist r, s G Q such that 

(ai, 02) = r(6i, 62) = s(ci, C2). 

Proof. Without loss of generality we can assume that Zi 7^ 0. Performing some elementary 
operations we get the system 

(ai^i — h\)li + (aiCi — c^)mi = 
(6ia2 - 6162)^1 + (cia2 - CiC2)mi = 
(6201 - &i&2)Zi + (c2ai - CiC2)mi = 0. 

Using that 6iCi 7^ the first two equations easily imply that aiC2 = Cia2- Since (oi — 
bi){ai — ci) 7^ the first and third equation easily imply that 61C2 = 62C1. The result 
follows. □ 

We can now prove the main result of this section: 

Proposition 3.7. Let pi,p2,p3 be essentially distinct polynomials and let pi =Pi—pi{0) 
for i = 1, 2, 3. Then: 

(z) W{pi,p2,P3) = 1 if and only if the polynomials pi,p2,P3 are linearly independent, 
(a) W{pi,p2,P3) = 3 if and only if some permutation of the polynomials Pi,P2,P3 has 
the form 

(a) {Ip, mp, kp^ + rp), or (b) {kp^ + Zp, kp^ + mp, kp^ + rp), 

for some k, l,m,r G Z. 

(Hi) In all other cases W{pi,p2,P3) = 2. 

Proof. We can assume that Pi{0) = for i = 1,2, 3. We first show part (i). Consider the 
standard Weyl system (T, T) of level 1 induced by the transformation Tx = x + a, where 
a G M is irrational. Let Xi{^) = e(rrijx), where rrii & Z for i = 0,1, 2, 3, be characters of 
T. Since the polynomials Pi,P2,P3 are linearly independent the equation 

Xoix) ■ XiiT'^^^'^x) ■ X2(TP^("^x) ■ XsiT^'^'^^x) = 1 

gives that mj = for z = 0, 1, 2, 3. By Proposition 13.21 we have that ^^(^1,^25^3) = 1- 

To show part (z) we first notice that since by Proposition [33] we have W{pi,p2,P3) < 3, 
it remains to show that W{pi,p2,P3) > 3 if and only if the polynomials have the form 
(a) or (6). To do this consider the quasi-standard Weyl system (T^,T) defined by 

T{xi, X2) = {xi + a,X2 + 2xi + a). 
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By Proposition 13.21 we have W{pi,p2,P3) > 3 if and only if there exist characters 
X05X15X25X3 of at least one of which depends nontrivially on the variable X2, such 
that 

(18) Xo{x) ■ XiiT^'^'^^x) ■ X2(T^'^^")x) ■ XsiT^'^^'^x) = 1 
for all X gT^. We use that 

T"(xi, X2) = (xi + na, X2 + 2nxi + n^a) 
and substitute in (ITHl) . We get that ^^(^1,^2,^3) > 3 if and only if the system 

(19) kipi + I1P2 + mip^ = 

(20) kipj + hpl + rriipl + /C2P1 + I2P2 + '"^2^3 = 

has an integer solution on the ki, li, rrii, i = 1,2, with at least one of the ki, li, mi nonzero. 

If the polynomial family has the form (a) then the following are eligible solutions to 
the previous system: (i) If 7^ then ki = mk, li = —kl, mi = 0, k2 = m{l — m)r, 
I2 = 0, m2 = ml{l — m), (ii) If k = then ki = rm{l — r), /c2 = rl{r — m), k^ = lm{m — l), 
k2 = 0, /2 = —r, m2 = m. Hence, W{pi,p2,P3) = 3. By Proposition 13.41 we get that the 
same is true for any polynomial family of the form (6). 

We now focus on the hardest part of the result which is to show that if W{pi,p2, ps) = 3 
then some permutation of the polynomials has either the form (a) or (6). Let 

Pi{n) = aiTi'^^ + . . . + ad^n, P2{n) = bin'^^ + . . . + bd^n, P3{n) = Cin'^^ + . . . + Cd^n 

for some di & N, i = 1,2,3, and ai,bi,Ci G Z with ai,bi,Ci 7^ 0. After rearranging the 
polynomials we can assume that di > d2 > d^. We consider the following three cases: 

Case 1. li di > d2 > d^ we will show that some permutation of the polynomials 
has either the form (a) or (6). From (fT9|) we get ki = 0, so P2,P3 are integer multiples 
of the same integer polynomial p. Using this and ( l20l) . we get that k2Pi is an integer 
combination of p and p"^. This easily implies that the polynomials have the form (a), 
possibly with some rational multiple of p in place of p. 

Case 2. li di > d2 > d3 we will show that pi = P2, a contradiction. From ( |T9l) we get 
that di = d2 = d and looking at the leading coefficients of the polynomials in ( |T9l) and 
(!20l) we get the system 

kiai + libi = 0, kial + lib\ = 0. 

Since ai, 61 7^ and ki, ^2 are not both zero, we easily get that ai = bi and ki + li = 0. 
If do = max{j G N: = 6j, for 1 < z < j} (it is well defined since Oi = 61), it suffices 
to show that do = d. Suppose not, then we can write pi = q + p'l, P2 = 9 + P2! where 
q{n) = ain^ + . . . + ad^n''-^'^^^ , the degrees d'l, d'2 of p'i,p'2 are not both zero, and they do 
not exceed do — k. By possibly permuting the polynomials we can further assume that 

(21) 4 < d[ < d, deg {p'l - p'2) = 
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Substituting pi = q + p'l, P2 = Q + and ki = — Zi in equations f|T9l) and fl20|) gives the 
system 



(22) 
(23) 



^iP'i - hP2 + mip^ = 

hPi - hp'i + t^ipI + '^hqip'i - p'2) + hPi + I2P2 + "^2^3 = 0. 



By ( 1221) we get nii 7^ (otherwise ki = nii = 0) and the polynomial p^ has degree at 
most d[. By fl2ip the polynomial q{p[ — p'2) has degree d + d'^ which is greater than the 
degree of all other polynomials that appear in (125]) . This can only happen if ki = 0, 
which gives mi = 0, contradicting our assumption that one of the integers ki,li,mi is 
nonzero. Hence, do = d which gives that pi = P2- 

Case 3. If di = d2 = d^ = d we will show that the polynomials have the form (6). 
We consider two subcases. Suppose that two of the three leading coefficients are the 
same, say for example that ai = bi (the other cases can be treated similarly). Then 
after replacing pi with qi = —pi, P2 with q2 = P2 — Pi, Ps with q^ = p^ — pi, and 
using Proposition 13. 4[ reduces our problem to either Case 1 or Case 2. Since Case 2 is 
impossible, the polynomials qi,q2,q3 have the form (a). It follows that the polynomials 
Pi,P2,P3 have the form (6) for some k 0. 

So it remains to deal with the case where all three polynomials have degree d and their 
leading coefficients ai,6i,Ci are distinct. In this case we will show that the polynomials 
have the form (6) with k = 0. The case where ci = 1 is trivial so we can assume that 
d > 2. There exist nonzero r, s G Q such that ai = rbi = sci. We will show by induction 
on t that for all 1 < t < d we have 

(24) (ai, 02, ... , at) = r(6i, 62, • • • , h) = s(ci, C2, . . . , q). 

The t = d case gives that the polynomials Pi,P2,P3 have the form (b) with k = 0. For 
t = 1 the statement is true by assumption. To better illustrate the idea of the inductive 
step we first work out the t = 2 case. Looking at the coefficient of n'^ and ra'^"^ in f|T9l) . 
and the coefficient of n^'^ and in (|20|) . we get (for d > 2 we have 2d — 1 > d) the 

system 



Since the integers cti, &i, Ci are nonzero and distinct we get by Lemma ES] that (ai, 02) = 
r{bi, 62) = s{ci, C2) for some nonzero r, s G Q, proving that (1241) holds for t = 2. 

Inductive step: Suppose that holds for some t G N with 1 < t < d, we will show 
that it holds for t + 1. So we need to establish that 



kitti + + rriiCi = 0, 
kial + libl + rriicl = 0, 



kia2 + hb2 + miC2 = 
kiaia2 + /1&1&2 + TniCiC2 = 0. 



(25) 



(Oi, ttt+i) 



r{bi,bt+i) = s(ci,Q+i). 
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Looking at the coefficient of ra'^ and n*^ * in (1191) . and the coefficient of rP'^ and n'^'^ * in 
f l20l) . we get the system 

(26) kiai + hbi + rriiCi = 0, hat+i + hh+i + "miQ+i = 

t 

(27) fcia^ + + micl = 0, ^(Ajifljat+s-j + hhA+2-i + miCiQ+2-i) = 0. 

i=l 

Since Oi = rhi = sci, the first equation in (1271) gives that 

(28) ki + hr-^ + mis'^ = 0. 

If 1 < i < t, by the inductive hypothesis we have = rbi = sCi. So for 2 < i < t (then 
t + 2 — i <t) we get 

kiaiat+2^i + libibt+2~i + miCiCt+2-i = aiat+2-i{ki + hr~^ + mis"^) = 0, 

where the last equality holds from fl28|) . This shows that in the second equation in (127|) 
all the terms in the sum except the ffist one are zero, hence 

kiaiat+i + hbibt+i + miCiQ+i = 0. 

If we replace the second equation in (1271) with this simpler one, Lemma 13.6! applies and 
gives ( l25l) . This completes the induction and the proof. □ 

4. Characteristic factors for the families {lip,l2P, ■ ■ ■ ,hp} and {pi,P2,P3} 

In this section we will prove Theorems A and B. 

4.1. Some preparatory work. We start with some preliminary results. We say that a 
collection V of integer polynomial families is eligible if whenever {pi{n), . . . ,Pk{n)} G V 
then (z) {pi(r?7, + s), . . . ,Pk{rn + s)} G P for every r G N, s = 0, ...,r — 1, and (ii) 
{cpi(?T,), . . . , cpk{n)} G V for every nonzero c G Q, as long as cpi G Z[t] for z = 1, . . . , fc. 

Proposition 4.1. Let V be an eligible collection of k-term polynomial families. Sup- 
pose that there exists an m ^ N such that for every totally ergodic nilsystem and every 
G V the factor is characteristic for weak convergence of the ergodic 
averages (P). Then the same is true for L"^ -convergence and for every ergodic system. 

Proof. We can assume that Pi{0) = for i = 1, . . . , k. By Theorem 11.21 we know that the 
averages (P) converge in L'^{fi), so the corresponding weak and strong limits coincide. 

Suppose that the factor Zm satisfies the assumption of the Proposition. It suffices 
to show that for every ergodic system (X, X, fi, T) if fi G L°°{fJ') fori = 1, . . . ,k and 
K{fi\Zm) = for some i = 1, . . . ,k, then the averages (P) converge to in L'^{fi) as 
N — M oo. Without loss of generality we can assume that i = 1. For ergodic systems, 
by Theorem 11.2! there exists a characteristic factor that is an inverse limit of nilsystems 
induced by some T-invariant sub-cr-algebras {Afjjjgpj- Since ]K{fi\Zm{X)) = implies 
that K{fi\Zm{Xj)) = 0, for j G Z, an approximation argument allows us to assume 
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that our system is an ergodic nilsystem, say (X = G/F, Q /T, m, To). By Proposition 12.11 
there exists an r G N such that the ergodic components of are totally ergodic. Since 
Pj(0) = 0, we have that Pi{nr) = rqi{n) for some integer polynomials qi, ior i = 1, . . . , k. 
Because V is eligible we have that {gi, . . . , g^} ^ 'P- We know from |26] that for every 
nonzero integer r and m G N we have Zm{Ta) = Zm{T'a). Since has finitely many 
ergodic components, it follows that if E(/|2m) = then the same holds for the ergodic 
components of T^. So using our assumption for the ergodic components of and the 
polynomial family {gi, . . . ,gfc}, we get that the averages (P) converge to in as 
A^ — M ^ cxD if we substitute Pi{rn) for pi{n) for i = 1, k. Finally, since E(/i|Zm) = 
implies that E(T^/i|^m) = 0, for j G N, a similar argument shows that the limit is also 
zero if we substitute pi{nr + s) for Pi{nr) in (P) for s = 0, . . . , r — 1. It follows that the 
averages (P) converge to in as — M — >• oo, completing the proof. □ 

Next we prove Theorem A. 

Proof of Theorem A. Let p be a nonconstant integer polynomial. We first claim that 
for totally ergodic systems the L^-limit of the ergodic averages associated to the fam- 
ilies {lip{n), hpin), . . . , lkp{n)} and {hn, l2n, . . . , Iku} are the same (a formula for the 
limit is then follows from Theorem 12. 9p . Using Theorem 11.21 and an approximation 
argument it suffices to check this for every totally ergodic nilsystem. So let (X = 
G/T, Q/T, m, To) be such a system. It suffices to show that for a.e. x G X the sequences 
{(a'i"x, a'2"x, . . . , a'*"x)}„gN and {(a'iP'^")x, a'^^^")^, . . . , a''=*'("')x)}„gN are equidistributed, 
or equivalently that the sequences {g{n)x}nm and {g{p{n))x}nm are equidistributed, 
where g{n)= (a'^"',a'2",. . . , a'*") is a linear sequence in and x = {x, . . . ,x) G X^. 
By Theorem 12.51 it is enough to show that for a.e. x G X the two sequences have the 
same closure. By Corollary 12.101 the set {g{n)x}j^^j^ is connected for a.e. x G X, so 
Proposition 12.71 applies and gives the required identity. 

We know from |19], [26] that the factor is characteristic for the family {hn, l2n,. . . , 
hn}, hence Zk-i is also characteristic for the family {lip{n), hpin), . . . , lkp{n)} for totally 
ergodic systems. Since the collection of polynomial families of the form {lip{n), hpin), . . . , 
hp{n)} with p G Z[t] nonconstant is eligible, by Proposition 14.11 the factor is also 
characteristic for every ergodic system. It was shown in |35] that 2a:-i is in fact the 
smallest characteristic factor for the family {/in, hn, . . . , hn}, the same argument shows 
that this is also the case for any family of the form {hp{n) , hp{n) , . . . , hp{n)} where 
p G 7j[t] is nonconstant. □ 

The next two lemmas will enable us to show that the Kronecker factor is characteristic 
for the averages (P) when k = 3 and W{pi,p2,P3) = 2. 

Lemma 4.2. Let ki, k2, h, h ^ be such that the polynomials kim, k2n, hm+hn are dis- 
tinct. Then for ergodic systems the Kronecker factor is characteristic for L"^ -convergence 
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of the averages 



N-l 



(29) 



{N - My 



m,n=M 



Proof. Let (X, A",/!, T) be an ergodic system and suppose that fi, f2, fs £ L°°{fi) with 
ll/illoo — ^ = 1) 2, 3. It suffices to show that if E(/j|/C) = for some i = 1, 2, 3 then 
the L^-hmit of the averages in (l29l) is zero. Suppose that E^f^lIC) = 0, the argument is 
identical if E(/2|/C) = Ojand if E(/i|/C) = we only have to interchange the role of m 
and n. By Theorem I2.5| M the L^-limit 



(30) 



does not depend on the choice of the F0lner sequence F/v- We claim that the L^-limit 
fl30l) is zero for the F0lner sequence 



(31) 



{0<m<N,0<n< a{N)}, 



where a{N) is an increasing sequence of integers that will be chosen later. We start by 
using the well known fact that for the family {an,bn}, where a,b are distinct integers, 
the Kronecker factor is characteristic (this is implicit in |T5]). Since E(/3|/C) = 0, we get 
that for every N & N there exists an a{N) G N such that 



(32) 



a(7V)-l 

^ ^ rpk2n _ r-pl2n ^rplim 



n=0 



1 

< — 

- N 



L2(m) 



for all m G {0, 1, . . . , — 1}. Furthermore, we can make sure that the sequence a{N) is 
increasing in A^. We have 



(33) 



^ \ ^ rpkim j rpk2n j rplim+l2n j 



< 



L2(m) 



1 ^ 

N ^ 



m=0 



a(7V)-l 



n=0 



Combining fl32l) and fl33|) , we get that for the choice of F0lner sequence made in fl3Tl) the 
L^-limit in fl30l) is zero, and so the same is true for the L^-limit of the averages fl29|) . □ 



More accurately, we have to combine Theorem 12.51 with a resuh in [26] that reduces the study of 
the hmiting behavior of hnear multiple ergodic averages along any F0lner sequence to nilsystems. 
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Lemma 4.3. Let {X, X , fi,T) be a totally ergodic system, pi,P2 be linearly independent 
integer polynomials, and ki,k2,li,l2^ ^ be such that the family P = {kipi, k2P2,hpi+hp2} 
has Weyl complexity 2. // /o, /i, /2, /s G L°°{fi') then the averages 

N-l 

^ ^ n=M 



and 
(35) 



(A^ - M)2 

^ ' n,r=M 



^ !( r ^ 



have the same limit as N — M ^ oo. 

Proof. By Theorem 11.21 and Lemma 14.21 there exists a factor of the system that is char- 
acteristic for both averages and is an inverse hmit of finite step nilsystems. So using 
an approximation argument it suffices to verify the lemma when the system is a totally 
ergodic nilsystem, say (X = G/T, Q/T, m, To). By Proposition 14. II the set X is connected 
so using Theorem 12.51 it suffices to show that for every x & X the sequences 

(36) {(a'^x, a'"+'=i"x, a'^+'^^'x, a"^+'i"+'2"x)}^,„,,6N 
and 

(37) {(a'"x, a^+^ifi^^^x, a'^+'^^^^^^^x, a^+'^^'i^^^+'^P^^")^)}^,^^^ 

have the same closure. 

Consider the subgroup of defined by 

H = {{9fi,gh\'f2,gh^2'h^9h'^h'^f,) e G'-. gMM e G, A, /a, /s, /4 e G2}, 

and A = fl F'^. Using part {Hi) of Theorem 12.51 we can show that the closure of the 
sequence in (!36|) is the connected nilmanifold H/A, where A = if fl (alternatively 
we can directly quote a more general result proved in Section 4 of [27j). It remains to 
show that the closure of the sequence in flTTI) is equal to H/A as well. To do this we 
are going to apply Theorem 12.51 First notice that if oq = (a, a, a, a), ai = (e, a^^^, e, a'^), 
a2 = (e, e, a'^^, a'^), and x = (x, x, x, x), then the sequences in fl5Bl) and fl57|) take the form 
{hi{m,n,r)x}m,n,rm and {h2{rn,n)x}m,n£n, where hi{m,n,r) = a[}*a"a2 and /i2(m, n) = 
a™a^^ 02^^"^ are polynomial sequences in H. 

Let vr: if/A ii'/([ifo, i^o]'^) be the natural projection. By Theorem 12.51 it suffices 
to show that for every x G X we have {h2{'m, n)7i{x)}^ = H/{[Hq, ifo]A). Since the 
sequence {/ii(m, n, r)7r(x)}m,n,r6N is dense in ii'/([ifo, -f^o]A) it is enough to show that 
for every x G X we have 



(38) {/i2(m, n)7r(x)}„ = {hi{m, n, r)7r(5)}^ „ 
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We first obtain some information about tlie quotient H / (\Hq, H^^^) . We claim tliat 
[Hq^Hq] = [GojCo]^. Tlie C inclusion is obvious. To establish the other inclusion first 
notice that for g E Gq elements of the form {g,g,g,g), {e,g^^,e,g^^), and {e,e, g''^, g^'^) 
belong to Hq. Taking commutators of these elements and using the fact that the group Gq 
is divisible, we easily get that [Go, Gq]'^ C [Hq, Hq], proving the claim. Hence, the quotient 
space H/{[Ho, Ho]A) can be identified with a subset of where Z = G/{[Go,Go]T), 
and so we can consider both sets in fl38|) as subsets of Z"^. 

Since Z is connected and the connected component of the group G/[Go, Gq] is abelian. 
Theorem 12.61 applies. So we can assume that Z = T'^ for some G N, tt{x) is represented 
by some {x,x,x,x) G T'^"', and the nilrotations by (a, e,e, e), (e, a,e, e), (e, e,a, e), and 
(e,e, e,a), are represented by the transformations S* x id x id x id, id x 5* x id x id, 
id X id X X id, id x id x id x 5*, where 5* is an ergodic nilpotent affine transformation of 
T'^. We have thus reduced our problem to showing that for every ergodic nilpotent affine 
transformation S acting on X = T'^, linearly independent integer polynomials pi,p2, and 
every x G T*^, the sequences 

S ( cm qm+kin qm+k2r qm+hn+hr \\ 

\\0 X, O X, O X, O X)}rn,n,reN 

and 

S ( Qm qm+kipi{n) qm+k2P2(n) qm+lipi{n)+l2P2(n) \\ 

have the same closure. Since 5* is uniquely ergodic the sequence {S"^x}m£^ is dense in X 
for every x E X. So it suffices to show that the sets 0{P, A^s, S) and 0{Q, Ax^, S) have 
the same closure, where Q is the family of 2-variable polynomials {kin, k2r,lin + l2r}. 
This in turn will follow if we show that the averages flMl) and fl35l) have the same limit 
as iV — M — > oo in the special case where the transformation T is equal to S. Since 
W{P) = 2, by Proposition 13.31 the characteristic factor for the averages flMl) when T = S 
is the Kronecker factor. By Lemma the Kronecker factor is also characteristic for the 
averages (|35ll . so it suffices to check the identity for group rotations. This can be easily 
verified for characters and then for general bounded functions by approximating them 
in by finite linear combinations of characters, thus completing the proof. □ 

4.2. Characteristic factors and limit formulas. We are now ready to prove Theo- 
rem B. The argument is rather lengthy so we refer the reader to the Introduction for 
a brief sketch. Notice that by Proposition 13.71 the cases (i), (ii), {Hi) of Theorem B 
correspond to the cases where the polynomial family has Weyl complexity 1, 3, 2 corre- 
spondingly. We deal with each one separately. 

4.2.1. Weyl complexity 1. Characteristic factor: We can assume that ~ ^^^^ 

i = 1, 2, 3. If the polynomials are linearly independent it was shown in [Tl] that the 

rational Kronecker factor ICrat is characteristic for L^-convergence of the averages (P). 
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Limit formula: In the case where the system is totally ergodic the factor ]Crat is trivial, 
hence for every /i, /2, /a G L°°{fJ') we have 
(39) 



n=M J J J 

where the limit is taken in LP'{^). 



4.2.2. Weyl complexity 2. Characteristic factor: The collection of 3-term polynomial 
families of Weyl complexity 2 is easily shown to be eligible, so by Proposition 14.11 we 
can assume that the system is totally ergodic. It follows from Proposition 13.71 that the 
polynomials {piip2iP^} are linearly dependent. Hence, 

{Pi,P2,P3) = {hqi + ci, k2q2 + C2,hqi + l2q2 + C3) 

for some linearly independent integer polynomials qi,q2 with zero constant term and 
ki, k2, h, h, Ci, C2, C3 G Z. Combining Lemmas 14.31 and Hl2l we get that for totally ergodic 
systems the Kronecker factor /C is characteristic for L^-convergence of the averages (P). 

It can be easily seen that for polynomial families of Weyl complexity 2 every charac- 
teristic factor (thought of as a subalgebra of functions) for the averages (P) contains all 
the eigenf unctions of the system, and as a result it contains the Kronecker factor. Hence, 
for ergodic systems the Kronecker factor is the smallest characteristic factor. 

Limit formula: We now compute the limit of the corresponding ergodic averages (P) 
for totally ergodic systems. We can assume that q = for i = 1,2,3. After replacing 
all three functions with their projection to the Kronecker factor JC we can assume that 
X = IC. Every Kronecker system is an inverse limit of 1-step nilsystems so we can assume 
that our system is a totally ergodic rotation on a compact abelian Lie group G with the 
Haar measure m. Moreover, by Proposition 14.11 the group G has to be connected, so 
Q — j^d some nonnegative integer d. In this case it is easy to check that for every 
/i, /2, /s G L°^{n) we have that 



(40) lim /i(T'=i^i(")t) ■ /2(r'^2«2(")t) ■ /3(T'i'?i(")+'2g2(n)^^ ^ 

°° n=M 

[ /i(t + kiti) ■ f2{t + k2t2) ■ fsit + hti + l2t2) dm{ti) dm{t2) 

for a.e. t G T*^. 

4.2.3. Weyl complexity 3. By Proposition 13.71 the polynomial triple (^1,^2,^3) either 
has the form (a) (/p-|-Ci,mp + C2, /^p^ + rp + cs), or (6) (A;p^ + /p + ci, k-p^ \ m]) \ 02-, kp^ + 
+ C3), for some integer polynomial p, and k, I, m,r, 01,02,03 G Z. We consider the 
following three cases: 
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Case 1: The family of essentially distinct polynomials has the form (a) with k = 0. 
This case is covered by Theorem A. The smallest characteristic factor is To find a 
limit formula in the totally ergodic case, first using standard deductions we can assume 
that the system is a totally ergodic 2-step nilsystem. In this case Theorem 12.91 gives a 
formula for the limit. 

Case 2: The family of essentially distinct polynomials has the form (a) with k ^ 0. 
We first deal with the case p{n) = n and then reduce the case of a general polynomial 
p{n) to this one. 

Characteristic factor for p{n) = n: Since the collection of polynomial families of the 
form (a) with /c 7^ is eligible, by Proposition 14.11 we can assume that the system is 
totally ergodic. Furthermore, we can assume that q = for i = 1,2,3. We first claim 
that if /s G /C"*- then the averages 

N-l 

(41) -3— Yl ^'Vi ■ T'^Va ■ T^"'+^"/3 



n=M 

'2/ 



converge to zero in L (fi) as N — M 00. We apply the Hilbert space Van der Corput 
Lemma (1^)0 for the sequence of functions 



a„(a;) = /i(T'"x) ■ /2(T™"a;) ■ /3(T'="'+™x). 



It suffices to show that for every h eN we have 

N-l 



— Tf ^ {an+h, an) 



lim 

N-M-^00 N — M 

n=M 

or equivalently that the average 

N-l _ 
^^2^ ^ ^ r-p{m—l)n ^rpmh j j ^ rpkn^ + {2kh+r)n ^rpkh'^+h j ^ rpkn^+rn j 

^ n=M 

converges to zero in L'^{fi) as — M — > 00. Using Proposition 13.71 it is easy to check 
that for all h, k,l,m,r E Z with h, k,l,m ^ and / 7^ m we have 

W{{m — l)n, kn^ + {2kh + r)n, kn^ + rn) = 2. 

Hence, as shown in the Weyl complexity 2 case, the characteristic factor for the ergodic 
averages (l42l) is the Kronecker factor. This proves the claim. 

Next we claim that if /i or /2 G A2 then the averages fj4Tl) converge to zero in 
L^(/i) as N — M — > 00. We prove this for /i, the argument is similar for /2. As we 
have shown, we can replace fs with Klf^lJC) without changing the limit of the averages 
fHTl) . Moreover, after approximating K{f^\}C) by a linear combination of eigenfunctions, 
and using linearity, we can assume that f^ is either constant, or a A-eigenfunction where 



11 



Let {a;„}rigN be a bounded sequence in a Hilbert space. If for every m G N one has 



liniTV-M^oo N-M '^n=M{^n+m,Xn) — 0, then Umw-M^oo N-M Sn=M -^n 



N-M 
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= 0. 



A = e{a) for some a G (0, 1). Moreover, since the system is totally ergodic a is irrational. 
If /s is constant the claim follows from a classical result of Furstenberg [15]. If not, the 
average PTI) is equal to /s times the average 



1 



N-1 

In _ 



(43) ^ r-fi ■ r-"/2 ■ e{{kn' + rn)a). 



n=M 



A simple computation shows that there exist characters xi!X2: — > C such that 

Xi(i?'"(ti,t2)) • X2(i?"^"(ti,t2)) = e{{kn^ + rn)a) 

holds for every n G N, where i?: is defined by 

RitiM) = {ti + (3,t2 + 2ti+(3) 

and (3 is some appropriately chosen rational multiple of a. Consider the product system 
(X X T"^ , fi X m, S = T X R) , where m is the Haar measure on T^, and let hi = fi ■ xi, 
^2 = /2 ■ X2- Then the average fH3l) takes the form 

N-1 

(44) j^-j^ ^'"^1 ■ ^"^"^2. 

n=M 

Let St, t G [0, 1], be the ergodic components of S. We will show that if /i G A2{T)-^ then 
hi{x) G K,{St)'^ for a.e. t. As it is well known, this would follow if we show that for a.e. 
X G X X we have 

I 1 

lim sup — / hi{S^ x) ■ e{ns) 
-^^°°se[o,i) I N ^ 



n=0 



0, 



or equivalently that 



N-1 

" ' 



1 1 

lim sup — > /i(T"'x) ■ e(ns + 72^7) 



e[o.i) „=o 

for a.e. x G X, where 7 is some integer multiple of (5. Since /i G A^iJ^^^ and T is totally 
ergodic this follows from [12]. Hence, h\{x) G )C{St)^ for a.e. t. From [15] we know that 
for distinct nonzero integers l,m the Kronecker factor is characteristic for the ergodic 
averages associated to the family {In, mn}. So an ergodic decomposition argument gives 
that the average in (jUj) converges to zero in L'^{n) as N — M —>■ 00, proving the claim. 
This shows that the factor A2 is characteristic for the averages (HTil . 

Limit formula for p{n) = n: We now compute the limit of the averages ( HTl) for totally 
ergodic systems. We can assume that q = for z = 1,2,3. Since A2 is a factor of 
Z2, and Z2 is an inverse limit of 2-step nilsystems, using an approximation argument 
we can assume that our system is a totally ergodic 2-step nilsystem that coincides with 
its 2-step affine factor A2- In this case the system is isomorphic to a 2-step nilpotent 
affine transformation on a connected compact abelian group G (see [1]). Furthermore, 
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our system is a nilsystem, so the group G has to be Lie. Hence, we can assume that G 
is a finite dimensional torus. In this case, the evaluation of the limit is a straightforward 
computation, which is done (for general /c-step affine systems) in [8| (or [27j). Instead of 
reproducing this rather complicated formula let us illustrate how the limit is computed 
in a simple case. Suppose that T : — > is given by 

where a is irrational and 6 is a nonzero integer. We find by direct computation that for 
a.e. (^1,^2) G we have 

(45) ^jim ^^^^-^ /i(r'"(ti,t2)) ■ /2(T™"(ti,t2)) • /3(r'="'+™(ti,t2)) = 

n=M 

fi{ti+lxi,t2+lyi+l^y2)-f2{h+mxi,t2+myi+m'^y2)-f3{ti+ky2+rxi,y3) dxi dyi dy2 dy^. 

•4 

It immediately follows from this formula (and more generally from the formulas in |8] or 
[27] ) that for almost every x E G the set of fH6|) is connected. 

Connectedness for p{n) = n: In order to deal with the case of a general polynomial 
p{n) we will apply Proposition 12.71 which allows as to make the substitution n —>■ p{n) 
when computing the orbit closure of a polynomial sequence with connected closure. We 
now verify that the connectedness assumption is satisfied, i.e. that for every totally 
ergodic nilsystem {X = G/T,Q/T,m,Ta) the set 

(46) H^ = { (a'"x, a"""x, a^"'+™x) }^^^ 
is connected for G X. By Theorem 12.51 we have 

N-l „ 

(4^) .r^T ^^r^E/i(«'"^)-/2K"^)-/3(«'"'+™a:)= / fl0f2®f3dmHx 

N-M—>oo iV — IVI ^ — ' I TT 

n=M ^ -"^ 

for a.e. x G X. Since the factor A2 is characteristic for convergence of the averages in 
(jUj), we can replace every function by its projection to A2 which by Proposition 12.41 is 
Z = G/{[Go,Go]G3T). This shows that the set factors through Z^. Furthermore, 
we know that acting on Z is topologically conjugate to a 2-step nilpotent affine 
transformation on some finite dimensional torus T'^. As we mentioned before, we can 
compute explicitly the limit in this case and derive that for a.e. x G X the projection of 
7:{Hx) of Hx onto Z^ is connected. It follows that the set Hx is a product of the connected 
set n{Hx) and the connected nilmanifold ([Go, G'o]G'3)/r', where F' = F fl ([Go; GojGs). 
Hence, for G X the set H^ is connected. 

General case: To deal with the general case notice that all the previous results carry 
through once we show that for totally ergodic systems the L^-limit of the averages in 
(H3l) remains the same if we replace n with any nonconstant polynomial p{n). Using 
Theorem 11.21 and an approximation argument, it suffices to verify that this is the case 
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for totally ergodic nilsystems. By Tlieorem l2.5l we can further reduce this to showing that 
if {X = G/r, Q/T, m, To) is a nilsystem with X connected, then for almost every x G X 
the sequences {(a'"-x, a™'"^, a^"^+™a;)}„eN and {(a'^^^^x, a"'^^"-)^, a''P(")^+'^P(")a;)}„eN have 
the same closure. We previously showed that for a.e. x & X the closure of the first 
sequence is connected. Hence, Proposition 12.71 applies and proves the claim. 

It can be easily seen that for polynomial families of the form (a) with A; 7^ every 
characteristic factor (thought of as a subalgebra of functions) for the averages (P) con- 
tains all the functions in £2 (defined in Section [2111) . and as a result it contains the factor 
A2- Hence, for ergodic systems the factor A2 is the smallest characteristic factor. 

Case 3: The family of essentially distinct polynomials has the form (6) with 7^ 0. 

Characteristic factor: It suffices to show that if G A2 for some i = 1,2,3 then the 
averages 



iV-l 

{ ) N — M ^ 



n=M 



converge to zero in L'^{fi) as N — M 00. We show this for i = 1, the argument is 
similar for i = 2,3. This time applying Van der Corput's lemma doesn't help. Instead, 
we notice that since the limit in L'^{fi) as X — M — > cxd of the averages (jUD exists, it 
suffices to show that if /i G A2 then for every /o G L'^ijj) we have 



1 ^"^ r 

\Wn \^ / f„ . J^kp(nf+lp(n) f . rpkp{n)'^+mp{n) r _ rpkp{nf+rp{n) f J _ n 

N-M^ooN-M ^ J Ji ■ 

^ T ^ 



n=M 

Equivalently we need to show that 

^ N-l 

lim 

N 



N-l 
n=M ^ 



for every /q G L°°{fi), which is true by Case 2. 

An argument analogous to the one explained in Case 2 shows that for ergodic systems 
the factor A2 is the smallest characteristic factor for the averages ( HHI) . Also a limit 
formula goes along the lines of Case 2. 



5. Applications in combinatorics 

In this section we are going to derive several combinatorial implications of our results 
in ergodic theory. Our starting point will always be the Correspondence Principle of 
Furstenberg that enables us to translate statements in combinatorics to statements in 
ergodic theory. We mention a slight modification of this principle due to Lesigne (see 
[6]) that allows us to work with ergodic systems (this is crucial for Theorem C): 
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Furstenberg's Correspondence Principle. For every A C N there exists an invertible 
ergodic system {X, X , fi,T) and A ^ X with fi{A) = d*{A) and such that 



d*(A n (A + ni) n . . . n (A + n^)) > fi{A n r"M n ■ ■ ■ n T"M), 

for all k and integers ni, . . . , n^. 

5.1. Sets of multiple recurrence. We will prove Theorem D. 

Proof of Theorem D. Suppose that p{n) is an integer polynomial that satisfies the as- 
sumptions of the theorem. Using Furstenberg's Correspondence Principle it suffices to 
show that if / € L°°{^) is nonnegative and not a.e. zero then 

1 r 

(49) lim — ^ / / ■ TP("V ■ . . . ■ T'^P^")/ dii > 0. 

n=0 

Using an ergodic decomposition argument we can assume that the system is ergodic and 
by Theorem 11.21 we can reduce the problem to showing (H9|) in the case where the system 
is an inverse limit of nilsystems. Moreover, an argument completely analogous to that 
of Lemma 3.2 in [T7] shows that the positiveness property (H9l) is preserved by inverse 
limits. Hence, we can further assume that the system is an ergodic nilsystem. In this 
case by Proposition 14 . 1 1 there exists an r G N such that the ergodic components of are 
totally ergodic. By our assumption there exists an no G N such that p^uq) = (mod r). 
Then p{rn + no) = rq{n) for some integer polynomial q and the limit in (l49l) is greater 
or equal than 1/r times 

lim — > 

N->oo N ^ 
n=0 

Using Theorem A for the ergodic components of T"^ we get that this last limit equals 

1 ^"^ /• 

lim - y / / ■ T"/ ■ • • • ■ T'^"/ d^i 

n=0 

which is positive by [15]. □ 

5.2. A bad set for recurrence with good powers. We will prove Theorem E. It 
will be a consequence of the Polynomial Szemeredi Theorem and the following multiple 
ergodic theorem: 

Proposition 5.1. Let {X, X , fi,T) be an invertible system, h: T ^ C be Riemann in- 
tegrable, /o,...,/^ G L°°{fj.), and P be an irrational number. Then for every integer 
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polynomial p with degp > 1 we have 



N-l 



(50) lim 1 ^ h{nf3) ■ / /o ■ T^^^Vi ■ • • • ■ T'^'^^Vfc dfi = 

n=0 

iV-1 
n=0 



Proof. Using an ergodic decomposition argument it suffices to check fl50|) when the system 
is ergodic. In [19j it is shown that for every A; G N if K{f\Zm{T)) = then E(/ (g) 
f\Zm^i{(T X T)t)) for a.e. t, where (T x T)t, t e [0, 1], denote the ergodic components 
of T X T. Keeping this in mind, and applying Theorem 11.21 for the ergodic components 
of the product system (X x X, X x X , fi x /i, T x T), we get that there exists an m G N 
such that if E{fi\Zm) = for some i = 0, . . . , k then 



D-hm / /o ■ T^'^^^fi ■ . . . ■ rf'=(")/fc dfi = 0. 



So (l50l) is obvious when E{fi\Zm) = for some i = 0, . . . , k, since then both limits are 
zero. We can therefore assume that fi G Z^ for i = 0, . . . , k. Since the factor Z^iT) is 
an inverse limit of nilsystems, a standard approximation argument shows that it suffices 
to check ( l50l) when the system is an ergodic nilsystem, say {X = G/r,Q/r,m,Ta). In 
this case, equation ( l50i) follows if we show that for /i, . . . , G L°°{fi) we have for a.e. 
X E X that 



(51) Jhn i 5^ /i(n/5) ■ /i(a^(")a;) ■ . . . ■ Ma'P^'^^x) = 

n=0 

/ /i(t) ■ jhn - 5^ /i(a^(")x) ■ . . . ■ Ma'^^'^^x). 

n=0 

By Theorem 12.51 it suffices to show that for G X we have 



(52) {(n/5, aP(")x, . . . , a'=P(")a;)}„gj^ = T x {(a^Wx, . . . , a'=P(")a;)}„gp^. 

Since degp > 1 this follows from Lemma [2.81 □ 

Proof of the Theorem E. We will show that the set 5 = {ra G N: {ny/2} G [1/4,3/4]} 
has the advertised property. Clearly 5* is not good for single recurrence since it is not 
good for recurrence for the rotation by \/2 on T. 

We will show that p{S) is a set of multiple recurrence whenever p is an integer poly- 
nomial with degp > 1. So let {X, X , fi,T) be an invertible system and A E X with 

n{A) > 0. We apply Proposition 15.11 for fi = 1^, i = 0,1,..., k, h = l[i/4,3/4], and 
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/? = v^. We get 



(53) 



lim — V fi(AnT~P^"^An 



ne5n[i,Af] 



1 1 

- ■ lim — V a(A n T-P^'^U n 

n=0 



The last limit is positive by Theorem ll.H showing that p{S) is a set of multiple recurrence. 



5.3. Universal families of three polynomials. We prove Theorem F. 

Proof of Theorem F. We can assume that the polynomials pi,p2,P3 are essentially dis- 
tinct. We claim that under the assumptions of the theorem, if / G L'^{ii) is nonnegative 
and not a.e. zero then 



An argument analogous to the one used in the beginning of the proof of Theorem D 
allows us to reduce the problem to showing ( |5^ in the case where the system is an 
ergodic nilsystem, say (X = G /V ,Q /V ,m,Ta). 

Weyl complexity 1. If the polynomials pi —p(0),j92 —P2(0),P3 —p3(0) are linearly in- 
dependent we have from Theorem B that the factor ICrat is characteristic for the averages 
in (15^ . hence we can assume that X = ICrat- Since our system is a nilsystem we have 
JCrat = foT somc r G N. By our assumption there exists Uq E N such that Piijio) = 
(mod r) for i = 1,2,3. Then pi{rn + no) = rp'-{n) for some integer polynomials p[ for 
i = 1,2, 3. Hence, whenever n = uq (mod r) we have T^»(") = id for i = 1,2, 3, and so 
the integral in (15^ is equal to f dfi > 0. The result follows. 

Weyl complexity 2. We start with some reductions on the polynomial family. We 
have that {pi,P2, Pz) = (^i^i + Ci, k2q2 + C2, hqi + l2<i2 + C3) for some linearly independent 
integer polynomials qi,q2 and ki,k2,li,l2,Ci,C2,C3 G Z. Since Pi{n) = (mod hi) has 
a solution for i = 1,2, we get that ci = kic[, C2 = /C2C2. So we are reduced to the 
case where the polynomial family has the form {kiqi, k2q2, hqi + hq2 + C3). If C3 7^ we 
can choose an r G N that is relatively prime to the integers ki, k2, C3. Then the system 
of equations Pi{n) = (mod r), i = 1,2,3, does not have a solution, contrary to our 
assumption. Hence, C3 = 0. So we can assume that 



for some linearly independent integer polynomials qi, q2 and ki,k2,h,l2 G Z. 

By Proposition 12. II there exists an r G N such that the ergodic components of are 
totally ergodic. By our assumption there exists no G N such that gj(no) = (mod r) for 



□ 



(54) 




{pi,P2,P3) = {kiqi,k2q2,hqi + hq2) 
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i = 1,2. Then qi{rn + Hq) = rq[{n) for some linearly independent integer polynomials 
q'l, ^he average in ( !54l) is greater or equal than 1/r times 



(55) liminf — ^ f- T^^^'^'i^")/ ■ T^'^^^^W/ . T^Ci^i^^^+'^^aW)/ f/^. 

n=0 

Working with the (totally ergodic) ergodic components of and the polynomial family 
{kiq'i, k2q'2, hq'i + hq'2} (which also has Weyl complexity 2) we get from Lemma that 
the limit in ( l55l) is equal to 



1 ^ r 

n,r=l 



which is easily shown to be positive. 

Weyl complexity 3. The argument is similar to the one used in the previous case 
so we just sketch the main steps. By Proposition 13.71 some permutation of the polyno- 
mials pi,p2,P3 either have the form (a) {Ip + ci,mp + C2, kp'^ + rp + C3), or the form 
(b) {kp"^ + lp + Ci, kp^ + mp + C2, kp"^ + rp + C3), for some integer polynomial p, and some 
fc, /, m, r, ci, C2, C3 G Z. Arguing as in the Weyl complexity 2 case, we can assume that 
Cj = for i = 1,2,3 and the system is totally ergodic. We consider the following three 
cases: 

In the case (a) with = we get from Theorem A that the limit in ( l54l) is equal to 



1 r 

Af^oo A* ^ — ^ / 
n=0 ^ 



which is positive by [15] . 

In the case (a) with k ^ we showed in the proof of part {ii) of Theorem B that the 
limit in is equal to 



N-l 

lim — V / / ■ T'"/ ■ T™"/ ■ T*="'+™/ rfu, 

n=0 



which is positive by Theorem 11.11 

To deal with the case (6) with A; 7^ we use the identity 

J j . 2^Pl(") J . 2^P2(n) J . rpP3{n) j! — J rp-P3{n) J! _ rppi{n)-p3{n) J! _ rpp2{n)-p3{n) j . j 

which allows us to show to reduce case (6) with A; 7^ to case (a) with /c 7^ that we 
previously handled. This completes the proof. □ 
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5.4. Positive results for lower bounds. The proof of Theorem C is an immediate 
consequence of Theorem C and Furstenberg's Correspondence Principle. So it remains 
to prove Theorem C. 

Proof of Theorem C. The proof for the case of two polynomials goes along the lines of 
the case of three polynomials with Weyl complexity < 2 and so we omit it. 

So let {pi,P2,P3} be a family of essentially distinct integer polynomials that is not 
equal to any of the exceptional forms mentioned in Theorem C. Then by Proposition 13.71 
the polynomial family either has Weyl complexity < 2, or some permutation of the 
polynomials has the form {kp, Ip, [k + for some integer polynomial p with p(0) = 
and k,l & Z. So we have to deal with the following two cases: 

Case 1. Suppose that W{pi,p2,P3) < 2. If W{pi,p2,P3) = 1 the polynomials are 
linearly independent and the result follows from [Tl]. If W{pi,p2,P3) = 2 we can assume 
that {pi,P2,P3) = {kiqi, k2q2,h(li + h(l2), where qi,q2 are some linearly independent 
integer polynomials and ki, k2, liih G Z. 

Suppose first that the system is totally ergodic. We can assume that its Kronecker 
factor has the form (G, Q, m, Rh) where G is a connected compact abehan group, Q is 
the Borel a-algebra, m is the Haar measure, and RhX = x + h for some b E G. If ^ is 
an open subset of G x G, define S = {n & N: {qi{n)b,q2{n)b) G V^}. We claim that if 
/i) f2, /a £ L°°{f^) are such that E(/j|/C) = for some i = 1,2, 3, then 

N-l 

(56) lim V ls(n) ■ T^^^iW/i . T'^^^aW j . r^hqiH+hq^in) j ^ q 

n=M 

where the limit is taken in We verify this as follows: First notice that since 

l5(n) = lv{qi{n)b, q2{n)b) and the function ly is Riemann integrable, using an approx- 
imation argument it suffices to show that (j56ll holds for ' X2{(l2{n)b) in place 
of ls{n), where Xi,X2 are any two characters of G. To see this consider the transfor- 
mation T' = T X Rb/k^ X Rf,/k2 acting on G^ with the Haar measure, where by b/k we 
denote a solution to the equation kx = b (since G is connected such a solution always 
exists). Let T/, t G [0, 1], be the ergodic components of T'. It is well known ([15]) that 
if E(/i|/C(T)) = then E{fi{x) ■ Xi(|/)|/C(T/)) = for a.e. t. Applying part (m) of The- 
orem B for the ergodic components of T' in place of T, and the functions fi{x) ■ Xi{y)j 
f2 ■ X2{z), /a in place of /i, /2, /s, we get the advertised identity. 
We will now apply ( 156|) for the set 

Ss = {neN: {qi{n)b,q2{n)b) G 5(0,5) x 5(0,5)} 

in place of S, where 5 > 0. First notice that since the transformation Rb is totally ergodic 

and the polynomials qi,q2 are linearly independent, the sequence {{qi{n)b,q2{n)b)}nm 
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is uniformly distributed in G x G. Hence, 

(57) lim ^4li^=(™(B(0.*)))l 

So (l56l) immediately gives that if /o, /i, /2, /s G L°°{fi) then 
(58) 

Urn / f„ . T'^i'Ji(") fi . T^2g2(n) / . 7^«igi(n)+;2g2(n) f _ 

N-T.oo\Ssn[M,N)\ ^ y ^° 

l;™ 3^ \^ / / pkiqiin) f 0^292 (n) ? p/ii?i(n)+;2q2{n) ? , 

where = E(/i|/C) for i = 0,l, 2, 3. 

We claim that the second limit in (l58ll is equal to 
(59) 

— — —2 / fo{9)-fi{9+ht)-f2{g+k2s)-f3{g+ht+ks) dm{g) dm{t) dm{s). 

{m{B{0,6))) JBio,s)xB(o,6) Jg 

One can verify this as follows: Since the sequence {{qi{n)b,q2{n)b)}neN is uniformly 
distributed in G x G, for every Riemann integrable function F: G x G ^ C one has 

1 r 

^60) ^Jr.^ 1^31^7 E ^(giW&,?2(ri)6) = / F(t,s) dm{t) dm{s). 

Applying (!60l) for 



F(t, s) = li?(o,5)xB{o,5)(i, s) ■ / 70(5-) ■ h{g + A^it) ■ f2{g + /C2s) ■ fz{g + /it + /2s) dm{g), 



G 



and using (1571) immediately gives that the limit in (1581) is equal to (l59l) . 

So we are left with estimating (1591) . for some appropriately chosen 5. First notice that 
if F : G X G ^ C is continuous then 

lim ^ ^/ F(t,s) dt ds = F(0,0). 

(m(S(0,5))) JB{0,5)xBiO,S) 

So if 5 is small enough, and fi = f = 1a, for i = 0, 1, 2, 3, the quantity in (159|) is greater 
than 

y (/y dm-e> (^j f dmY -e = fi{AY - e. 

Summarizing, we have shown that if ^^(^1,^2,^3) < 2 and the system {X, B, fj,,T) is 
totally ergodic, then for every e > 0, if 5 is small enough we have 

N~M^oo SsD M,N) ^ '-^^ ' 

' ^ ' ^' n&Ssn[M,N) 

This completes the proof of Case 1 for totally ergodic systems. 
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In the general case, since the Kronecker factor /C is an inverse hmit of systems with 
finite rational Kronecker factor ICrat-, we can choose r G N and a factor K,' of /C such that 
/C' n Krat = K,r and 



Then up to an error term e, equation (!58|) remains valid after replacing /j with E(/j|/C'), 
for i = 0,1, 2, 3. The system (/C', T) is isomorphic to an ergodic rotation on Hx'Zr, where 
if is a connected abelian group. We write qi{rn) = rq[{n), i = 1,2, for some integer 
polynomials qi,q2, and work with T*" in place of T and q'iin) in place of qi{n), i = 1,2. 
Arguing as in the totally ergodic case we get the desired lower bound, completing the 
proof of Case 1. 

Case 2. Suppose that some permutation of the essentially distinct polynomials has 
the form {lp,mp, {I + m)p} for some integer polynomial p with p{0) = and l,m & Z. 
Our tactic will be similar to the one used in the previous case but extra complications 
arise because the relevant characteristic factor in this case is not "abelian" . 

Suppose first that the system is totally ergodic. Using an approximation argument we 
can assume that the factor Z2 is an ergodic 2-step nilsystem, say (X = G/T, Q/T, m. To). 
By Proposition l4.1l we have that X is connected. Since G is 2-step nilpotent, the subgroup 
r2 = G2 n r is normal in G. So G/V2 is a group and X = {G/T2)/ (T/T2). Using this 
representation for X we can assume that r2 = {e} and so G2 is a compact abelian Lie 
group. Since G2 is connected we can further assume that it is a finite dimensional torus 
with the Haar measure A. Likewise, Z = X/[G, G] is a connected compact abelian group 
and so we can assume that it is a finite dimensional torus with the Haar measure A'. 

If tt: X — > Z is the natural projection, and V is an open subset of Z, let S = {n & 
N: p{n)aQ G V} where Oq = 7r(ar) (we use additive notation on Z). We first claim that if 
/15 /2, /s G L°°{fJ') are such that E(/j|Z2) = for some i = 1,2, 3, and /, m, r are distinct 
nonzero integers, then 



where the limit is taken in LF'{^). We verify this as follows: We can assume that the 
integers l,m,r are relatively prime (if not we write I = l'd,m = m'd,r = r'd where 
d = gcd{l,m,r), and work with the polynomial family {I'p' ,m'p' ,rp'} where p' = dp). 
Hence, there exist /i,mi,ri G Z such that //i+mmi + rri = 1. Since ls{n) = lv{p{n)ao) 
and the function ly is Riemann integrable, using an approximation argument it suffices 
to verify ( |6T1) with x{p{^)(^o) in place of ls'(r;,), where x is any character of Z (using 
our notation we have x(a^*^"'''r) = x(p(n)ao)). This last statement follows immediately 
by applying Theorem A for the functions /i ■ x{hg), f2 ■ x{^i9)^ /s ' xi^id) in place of 
/i, /2, /s- 



||E(/|r)-E(/|/C)||^. 



< e/3. 




N-l 



(61) 



n=M 
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We will now apply fl6T|) for the set 

5^ = G N: p{n)ao G B{0,6)} 

in place of S, where 6 > 0. First notice that since the sequence p{n)ao is uniformly 
distributed in Z we have that 

So (pTl) immediately gives that for /o, /i, /2, /s G L°°{fi) we have 

' ^ ' ^' n&Ssn[M,N) 



' ^ ' ^' neSsn[M,N) 

where /j = E(/j|Z2) for i = 0, 1, 2, 3. We claim that the second limit in (163|) is equal tc0 

(64) —TT^^^J [ [ foigT)-h{gg[gPT).UggTgPT). 
m{B{G2,6)) Jx Jb{G2,5) Jg^ 



fz{gglg2^) d\{g2) dm{giT) dm{gT), 

where B{G2,S) = '7r~^(i?(0, 5)). This can be seen as follows: Since X is connected, we 
can use the formula of Theorem 12.91 with p{n) in place of n (by Theorem A), x, /i, /2, /s 
in place of fi, fk, fi, fm, and 1 in place of all other /j, where x is any character of Z. We 
get that for a.e. x = gT E X we have 



N-l 

(65) ^ jim ^ j^-j^ Yl x{p{n)ao) ■ /i(a'^(")x) ■ Ma^^^^^^x) ■ h{a^^^^^x) 



n=M 




x{giT) ■ fMig^^-^) ■ f2{ggTgl''T) ■ h{gg{g\''T) dX{g2) dm{g,T). 

IX JG2 

Using an approximation argument we can verify that (1^ holds with 1b{G2,&) place of 
X- If we multiply this last identity with folgT), then integrate with respect to m{x) and 
use (1^ . we get that the limit in (jSSD is equal to proving our claim. 

So we are left with estimating fl64p for r = I + m and some well chosen 6 > 0. It 
suffices to show that when all functions are equal to / = 1^ the limit of 0641) as 5 — » 
is greater or equal than /i(A)^. Since 7r^^(0) = (G'2r)/r — G2 it is not hard to see that 
this limit is equal to 

C) _ f"^'] _ (l + m\ 

fm ■ f{gg[gr'r) ■ f^gg^gr^V) ■ figgi^'^gl ' ^V) dX{g2) dX{g,) dm{gV). 




X JG2XG2 



^^It may not be immediately obvious but the next integral is well defined. For more details see [34] . 
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Since the elements of G2 commute with all the elements of G we can write the last 
integral as 

(66) / / fix) ■ f{g[gPx) ■ figTgPx) ■ f [gi^"^ g^'^K) dX{g,) dX{g,) dm{x). 

Jx JG2XG2 

It will be more convenient to rewrite (166|) as 
(67) 

/ / f\9xyhgg[gPxyf{ggTgPxyf{gg[-'"'gP'^K) dX{g) d\{g,) dX^g^) dm{x) 

Jx JG2XG2XG2 

An easy algebraic manipulation shows that the set 

{{9.99\9\'\99T9r\99[^"'9\ ' l'- 9.9i.92 G G,} 

is equal to the set 

{(/ii,/i2,/i3,M e Gl: hT''h'+"' = /ir'/^2^™}l3 
So the integral fl67|) can be rewritten as 

( f{hix) ■ f{hzx) dX{hi,h^) \ dX{h) dm{x). 

Jx Jg2 ^ Jh'^-'hi;;+'=h ' 

Using Cauchy-Schwarz and a change of variables, we see that the last integral is greater 
or equal than 

N 4 

fihx) dXiK) ) dm{x), 



IX ^JG2 ' 

which is greater or equal than 

^ / / ^^^^^ dm[x)^^ = (^j f{x) dm{x)y = fx{A)^. 

This completes the proof for totally ergodic systems. 

In the general case, since every nilsystem is an inverse limit of nilsystems with finite 
rational Kronecker factor Krat, there exists tq G N and a factor 3^ of our system, such 
that 3^ is a nilsystem, y fl JCrat = y 

||E(/|3^)-E(/|Z2)||^.(^)<e/10. 

Then up to an error term e equation fl63l) remains valid after replacing /j with E(/j|3^), 
for i = 0,1,2,3. Moreover, the ergodic components of the system {y,T^'°) are totally 
ergodic. We write p{ron) = rog(n) for some integer polynomial q and work with T^'^ in 
place of T and q{n) in place of p{n). Arguing as in the totally ergodic case we get the 
desired lower bound. □ 



^'^The symmetry of this equation is what allows us to obtain the required lower bounds. This sym- 
metry is violated when r ^ I + m making it rather unlikely that similar lower bounds hold in this case. 



We discuss this more in Section 1531 
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5.5. Conditional counterexamples for the exceptional cases. We explain why we 
expect the lower bounds of Theorems C and C to fail for the exceptional polynomial 
families (ei) with I < m < r and r 7^ l+m, (62), (es). To avoid unnecessary complications 
we will work out the details for two typical cases, the general case can be treated in a 
similar fashion. 

We first review a notion defined by Ruzsa in [31] . We consider equations of the form 

(68) aiXi + . . . + ttkXk = 0, 

where ai, . . . , G Z satisfy ai + . . . + = 0. Let Aat be the subset of maximum 
cardinality of {1, ... , N} that does not contain solutions to (|68l) with k distinct entries. 
We define the type of the equation (168|) to be the number 

r r log|A7v| 
1 =limsup—; — . 

N^oo log A/ 

For example the equation ax + by = az + bw with a,b ^ has type 1/2 (see [31]), and 
an example of Behrend [5] shows that the equations x + y = 2z,x + y + z = 3w have 
type 1. It seems plausible that the equation 

(69) ax + by = cz + dw, a,b,c,d > 0, a c,d, a + b = c + d, 

has always type 1. In support of this, very recently it was shown in [21] that "most" 
equations of the form flB^ have type 1 (for example the equation 3x + y = 2z + 2w). 
It also seems plausible that any equation in five variables (in fact in any odd number of 
variables) with nonzero coefficients has type 1. 

Proposition 5.2. (i) Suppose that the equation x + 8z = 6y + 3w has type r0 Then 
for every 6 <r there exists an ergodic system {X, X , fi,T) and A E X such that 

n{A n r-2"A n r-^"^ n t~^''A) < ^(A)" 

for every G N, where c = Irf (notice that c ^ 00 as 5 ^ 1). 

(ii) Suppose that the equation 2x + y + w = 2z + 2v has type T. Then for every S < T 
there exists an ergodic system {X, X , fi,T) and Ae X such that 

^i{A n T~"A n t-^m n t-"'a) < ^(A)"^ 

for every n G N, where d = ^ ■ (notice that d ^ 00 as 5 1). 

Proof, (i) On X = with the Borel a-algebra B and the Haar measure /i = A x A, 
define the ergodic measure preserving transformation T : — by 

T{t,s) = {t + a,s + 2t + a), 

for some irrational a. One then easily finds that 

T'^^t, s) = {t + na, s + 2nt + r?a) 



^■^This equation is not covered by the results of |2T], so it is not yet known whether it has type 1. 
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for every n G N. Let A = T x B E B, where the set B will be chosen later. We compute 

(70) fi{A n T^^A n T^"A n t^"A) = 

/ leis) ■ 1b{s + Ant + An^a) ■ Isis + 6nt + 9n'^a) ■ Isi'S + 8nt + IQn'^a) d\{s) d\{t). 

Notice that the four numbers 

X = s, y = s + Ant + An'^a, z = s + 6nt + 9n'^a, w = s + 8nt + IGn^a 
satisfy the equation 

(71) X + 8z = 6y + 3w. 

By our assumption, for every 5 < F, there exist sets An C {1, . . . , A^}, such that |AAr| ^ 
and Ajv contains no solution to fl7T]) with distinct entries. Let 

ieAiv 

Because of the condition on A at it can be easily verified that if x,y, z,w & B satisfy (ITT]) 
then at least two of the x,y,z,w belong to the same subinterval 1^ = ^ + gj^y) 
for some jo G Aat. Say for example that x,y are these two elements. We get that 
Ant G [ — An'^a — -^j^, —An^a + -^jf) and so t belongs to a set of measure at most 
2/(81A^). The other five cases give a similar condition, so t belongs to a set In,N of 
measure at most 12/(81A^) < Hence, the integral in fITU]) is at most 

1b{s) dX{s) dX{t) = X{B) ■ A(4,^) < 
Since lA^rl ^ A^'', for c = Izf easy computation shows that 



) =KAy 



m \81N. 

By choosing A^ large enough we get the advertised estimate. 

{ii) Let (X, T) be the system used in (i) and \et A = B x B eB, where the B 
will be chosen later. We find 

(72) iJi{AnT'AnT'^''AnT'' A) < 

Isit) ■ 1b{s) ■ 1b{s + 2nt + n^a) ■ 1b(s + Ant + An^a) ■lB{t + n^oi) dX{s) dX{t). 

The five numbers 

X = t, y = s, z = s + 2nt + n^a, w = s + Ant + An'^a, v = t + r?Oi 
satisfy the equation 

(73) 2a; + y + w = 2z + 2w. 
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By our assumption, for every 5 < F, there exist sets A^v C {1, . . . , N} such that lAjvl ^ 
A^'' and Aat contains no solution to ( ff3|) with distinct entries. By [3j, there exist sets B^^f C 
{1, . . . , A^} with \Bn\ ^ JVg-cViogJV j^^ contains no nontrivial 3-term arithmetic 
progressions. An averaging argument shows that there exists n = n{N) G {1, . . . , A^} 
such that the set A'^ = H {B^ + n) has cardinahty |A'^| > |AAr| ■ |i?Ar|. Then 
|A'^| A^*^' whenever 6' < 6. Hence, for every 5 < F there exists a set A^r C {1, . . . , A^} 
with the following properties: lA^vl ^ A^*^, Ajy contains no solution to fl73|) with distinct 
entries, and A^r contains no nontrivial 3-term arithmetic progressions. Let 

Because of the condition on A^, it can be easily verified that if x,y,z,w,v G B satisfy 
(1731) . then at least two of the x,y, z,w,v, excluding the pair x,v, belong to the same 
subinterval ^ + j^) for some jo ^ A^v- It follows that the integral in (1721) is 
bounded by the sum of (2) — 1 = 9 integrals, one of which (a typical one) has the form 



Ix,z = / lBit)-lBis)-lBis + 2nt + n'^a)-lBis + Ant + An'^a)-lBit + n^a) dX{s) dX{t), 
where 

Ax,z = {{t,s) eT'^: X = t,z = s + 2nt + r?a G Ij = [J^, ^ + Y^) ^ ^ ^^}- 



We have 



Ix,z < /, l^.x^.(^' s + 2nt + n^a) d\{s) d\{t). 



Since the transformation (t, s) ^ (t,s + 2nt + n?a) acting on is measure preserving, 
this leads to the estimate 

\2 \^n\ 



h,z < ^(^.■ 



Similarly, we find the same bound for the other 8 integrals. Combining all 9 integrals we 
get that 



2n /I T^n A\ ^ 



ATS 

Since |AAr| ^ A^'', for = | ■ Izf an easy computation shows that 



2d 



iV2 



By choosing A^ large enough we get the advertised estimate. □ 



We derive an analogous result in combinatorics: 
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Proposition 5.3. (z) Suppose that the equation x + 8z = 6y + 3w has type T. Then for 
every 6 < T there exists A C N such that 

d*{A n (A + 2n) n (A + 3n) n (A + An)) < d*{A)'' 

for every n G N, where c = Izf ■ 

(a) Suppose that the equation 2x + y + w = 2z + 2v has type T. Then for every 6 < T 
there exists A C N such that 

d*{A n (A + n (A + 2n) n (A + n^)) < d*{A)'^ 

for every n eN, where d = ^ ■ ■ 

Proof. Let {X, X, fi, T) be the system and A E X he the set used in the proof of Propo- 
sition I5.2[ Fix an xq E X and let 

A = {n G N: T"xo G A}. 

Since the system is uniquely ergodic and the boundary of the set A fl T^^ A fl ■ ■ ■ fl T"* A 
has measure zero, we have that 

, , ^„ , ^„ .X , \{mE\M,N):T"'xEAr\T''^Ar]---nT''^A}\ 

uM nT"Mn---nT"'=A = lim ^ ^— — — — ^ 

'^^ ^ Ar_M-»oo N - M 

= d*{A n (A + rai) n • ■ ■ n (A + Uk)) 

for every ni, . . . ,nk E N. The result follows from Proposition 15.21 □ 

From the previous results we conclude that if the type of the equation x + 8z = 6y + 3w 
is greater than 2/3, or the type of the equation 2x + y + w = 2z + 2v is greater than 6/7, 
then the lower bounds of Theorems C and C fail for the families {2n, 3n, 4n}, {n, 2n, n^} 
correspondingly. If the type of both equations is 1 then they fail for any fixed power of 
or d*{A). 

All the other exceptional families of Theorems C and C can be treated similarly. 
Polynomial families of the form (ei) with I < m < r and r ^ I + m lead to equations of 
the form flB^ . and polynomials families of the form (62), (63) lead to equations in five 
variables. Unfortunately, none of these equations can be treated using the results in [21] . 
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